Oct 29 22:30:39.073355 ip-10-0-136-228 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 29 22:30:39.073367 ip-10-0-136-228 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 29 22:30:39.073373 ip-10-0-136-228 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 29 22:30:39.073682 ip-10-0-136-228 systemd[1]: Failed to start Kubernetes Kubelet. Oct 29 22:30:49.103229 ip-10-0-136-228 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 29 22:30:49.103244 ip-10-0-136-228 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 54af53cc5e9746bdb7ce40762eb8adfd -- Oct 29 22:32:41.263149 ip-10-0-136-228 systemd[1]: Starting Kubernetes Kubelet... Oct 29 22:32:41.774884 ip-10-0-136-228 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 22:32:41.774884 ip-10-0-136-228 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 29 22:32:41.774884 ip-10-0-136-228 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 22:32:41.774884 ip-10-0-136-228 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 22:32:41.779323 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779245 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.779323 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779323 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.779323 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779328 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779331 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779334 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779337 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779339 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779342 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779344 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779348 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779352 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779357 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779360 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779363 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779365 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779368 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779370 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779373 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779376 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779378 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779381 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.779416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779384 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779386 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779389 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779391 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779394 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779396 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779400 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779402 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779404 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779407 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779409 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779412 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779415 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779417 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779420 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779423 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779426 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779428 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779431 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779434 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.779874 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779436 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779439 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779441 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779444 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779446 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779448 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779451 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779453 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779455 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779458 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779460 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779463 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779465 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779468 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779472 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779475 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779477 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779480 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779482 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.780453 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779485 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779488 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779490 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779493 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779495 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779497 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779500 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779502 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779505 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779507 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779510 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779512 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779515 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779517 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779519 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779522 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779524 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779527 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779529 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779532 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.780904 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779534 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779536 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779539 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779541 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779544 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779546 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779549 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779552 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779555 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779558 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779561 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779563 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779567 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779572 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779575 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779578 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779580 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779583 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779586 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.781390 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779588 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779590 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779593 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779595 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779599 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779601 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.779603 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780436 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780443 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780447 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780450 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780452 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780456 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780459 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780461 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780464 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780467 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780469 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780473 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780476 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.781826 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780478 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780483 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780486 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780489 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780492 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780495 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780497 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780508 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780512 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780516 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780518 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780521 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780523 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780526 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780529 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780531 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780534 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780536 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780539 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.782307 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780541 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780544 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780546 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780549 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780551 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780554 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780556 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780559 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780561 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780563 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780566 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780570 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780572 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780575 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780578 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780580 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780583 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780585 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780588 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.782756 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780590 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780593 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780601 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780604 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780606 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780609 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780611 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780614 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780616 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780619 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780621 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780624 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780626 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780629 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780631 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780634 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780636 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780639 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780641 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.783216 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780644 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780646 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780649 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780651 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780654 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780657 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780660 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780663 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780666 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780668 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780671 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780673 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780676 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780678 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780681 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780683 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780691 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780693 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780696 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780698 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.783669 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780701 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780703 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780706 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780708 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780711 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780713 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780715 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780718 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780720 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780722 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780725 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780728 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780731 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780733 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780736 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.780745 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781325 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781332 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781335 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781338 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.784146 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781341 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781343 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781346 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781349 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781351 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781354 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781357 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781359 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781367 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781370 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781373 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781375 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781378 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781380 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781383 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781385 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781388 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781390 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781393 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.784635 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781396 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781398 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781401 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781404 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781406 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781409 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781415 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781417 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781420 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781423 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781426 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781430 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781433 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781436 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781438 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781441 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781443 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781446 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781449 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.785106 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781451 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781453 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781456 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781464 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781467 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781469 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781472 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781474 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781477 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781479 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781482 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781484 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781487 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781489 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781491 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781494 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781496 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781499 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781501 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781505 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.785555 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781508 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781510 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781514 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781518 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781521 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781523 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781526 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781528 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781531 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781533 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781536 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781538 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781541 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781543 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781545 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781548 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781557 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781560 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781562 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781565 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.786077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781567 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781570 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781572 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781575 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781577 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781579 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781582 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781586 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781590 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781593 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781596 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781600 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781602 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781605 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781607 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781611 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781613 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781615 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781618 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781620 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.786537 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781623 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781625 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781628 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.781631 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782503 2574 flags.go:64] FLAG: --address="0.0.0.0" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782515 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782525 2574 flags.go:64] FLAG: --anonymous-auth="true" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782529 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782534 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782543 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782548 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782552 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782555 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782558 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782561 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782565 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782568 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782571 2574 flags.go:64] FLAG: --cgroup-root="" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782573 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782577 2574 flags.go:64] FLAG: --client-ca-file="" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782580 2574 flags.go:64] FLAG: --cloud-provider="external" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782583 2574 flags.go:64] FLAG: --cluster-dns="[]" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782589 2574 flags.go:64] FLAG: --cluster-domain="" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782595 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 29 22:32:41.787002 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782598 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782601 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782604 2574 flags.go:64] FLAG: --container-log-max-files="5" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782609 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782612 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782614 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782618 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782621 2574 flags.go:64] FLAG: --contention-profiling="false" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782624 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782626 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782629 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782632 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782637 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782640 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782643 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782645 2574 flags.go:64] FLAG: --enable-load-reader="false" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782648 2574 flags.go:64] FLAG: --enable-server="true" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782651 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782663 2574 flags.go:64] FLAG: --event-burst="100" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782666 2574 flags.go:64] FLAG: --event-qps="50" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782669 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782672 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782675 2574 flags.go:64] FLAG: --eviction-hard="" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782679 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 29 22:32:41.787569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782682 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782685 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782688 2574 flags.go:64] FLAG: --eviction-soft="" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782691 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782694 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782698 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782701 2574 flags.go:64] FLAG: --experimental-mounter-path="" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782705 2574 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782708 2574 flags.go:64] FLAG: --fail-swap-on="true" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782711 2574 flags.go:64] FLAG: --feature-gates="" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782715 2574 flags.go:64] FLAG: --file-check-frequency="20s" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782718 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782721 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782725 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782728 2574 flags.go:64] FLAG: --healthz-port="10248" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782731 2574 flags.go:64] FLAG: --help="false" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782734 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-136-228.ec2.internal" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782737 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782740 2574 flags.go:64] FLAG: --http-check-frequency="20s" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782743 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782746 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782750 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782753 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 29 22:32:41.788159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782756 2574 flags.go:64] FLAG: --image-service-endpoint="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782759 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782762 2574 flags.go:64] FLAG: --kube-api-burst="100" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782765 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782772 2574 flags.go:64] FLAG: --kube-api-qps="50" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782775 2574 flags.go:64] FLAG: --kube-reserved="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782778 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782781 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782784 2574 flags.go:64] FLAG: --kubelet-cgroups="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782787 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782790 2574 flags.go:64] FLAG: --lock-file="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782792 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782795 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782798 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782804 2574 flags.go:64] FLAG: --log-json-split-stream="false" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782807 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782810 2574 flags.go:64] FLAG: --log-text-split-stream="false" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782813 2574 flags.go:64] FLAG: --logging-format="text" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782816 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782819 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782822 2574 flags.go:64] FLAG: --manifest-url="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782825 2574 flags.go:64] FLAG: --manifest-url-header="" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782829 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782832 2574 flags.go:64] FLAG: --max-open-files="1000000" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782847 2574 flags.go:64] FLAG: --max-pods="110" Oct 29 22:32:41.788703 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782850 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782853 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782855 2574 flags.go:64] FLAG: --memory-manager-policy="None" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782858 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782861 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782864 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782866 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782874 2574 flags.go:64] FLAG: --node-status-max-images="50" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782877 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782879 2574 flags.go:64] FLAG: --oom-score-adj="-999" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782882 2574 flags.go:64] FLAG: --pod-cidr="" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782885 2574 flags.go:64] FLAG: --pod-manifest-path="" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782893 2574 flags.go:64] FLAG: --pod-max-pids="-1" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782897 2574 flags.go:64] FLAG: --pods-per-core="0" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782900 2574 flags.go:64] FLAG: --port="10250" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782902 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782905 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d56abb05ac7d75ce" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782908 2574 flags.go:64] FLAG: --qos-reserved="" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782911 2574 flags.go:64] FLAG: --read-only-port="10255" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782914 2574 flags.go:64] FLAG: --register-node="true" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782916 2574 flags.go:64] FLAG: --register-with-taints="" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782920 2574 flags.go:64] FLAG: --registry-burst="10" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782923 2574 flags.go:64] FLAG: --registry-qps="5" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782925 2574 flags.go:64] FLAG: --reserved-cpus="" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782928 2574 flags.go:64] FLAG: --reserved-memory="" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782932 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 29 22:32:41.789304 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782935 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782938 2574 flags.go:64] FLAG: --rotate-certificates="false" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782941 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782944 2574 flags.go:64] FLAG: --runonce="false" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782947 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782950 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782952 2574 flags.go:64] FLAG: --seccomp-default="false" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782955 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782958 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782961 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782964 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782967 2574 flags.go:64] FLAG: --storage-driver-password="root" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782970 2574 flags.go:64] FLAG: --storage-driver-secure="false" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782972 2574 flags.go:64] FLAG: --storage-driver-table="stats" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782975 2574 flags.go:64] FLAG: --storage-driver-user="root" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782978 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782982 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782985 2574 flags.go:64] FLAG: --system-cgroups="" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782987 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782994 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782997 2574 flags.go:64] FLAG: --tls-cert-file="" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.782999 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783006 2574 flags.go:64] FLAG: --tls-min-version="" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783008 2574 flags.go:64] FLAG: --tls-private-key-file="" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783011 2574 flags.go:64] FLAG: --topology-manager-policy="none" Oct 29 22:32:41.790317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783014 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783017 2574 flags.go:64] FLAG: --topology-manager-scope="container" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783019 2574 flags.go:64] FLAG: --v="2" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783024 2574 flags.go:64] FLAG: --version="false" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783028 2574 flags.go:64] FLAG: --vmodule="" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783032 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783036 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783207 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783211 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783214 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783217 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783220 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783222 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783225 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783228 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783231 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783234 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783236 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783239 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783241 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783244 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.791432 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783246 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783249 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783251 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783253 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783256 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783258 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783266 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783269 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783271 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783274 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783276 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783279 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783281 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783284 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783286 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783289 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783291 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783295 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783298 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783300 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.792336 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783303 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783305 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783308 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783310 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783313 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783315 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783318 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783321 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783323 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783326 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783328 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783330 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783333 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783335 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783338 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783340 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783343 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783345 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783348 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783356 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.793212 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783359 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783361 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783363 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783366 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783368 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783371 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783373 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783377 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783380 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783384 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783386 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783389 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783391 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783394 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783397 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783399 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783402 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783404 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783407 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.794095 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783410 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783412 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783417 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783420 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783423 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783425 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783428 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783430 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783433 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783436 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783438 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783440 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783443 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783453 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783457 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783460 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783463 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783465 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783468 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.794664 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783470 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783474 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783477 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783480 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783483 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783485 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783488 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783490 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783493 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783495 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783498 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783500 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783503 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783505 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.783532 2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783613 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.795396 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783628 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783632 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783636 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783639 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783642 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783645 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783648 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783651 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783654 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783656 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783659 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783672 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783675 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783678 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783681 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783683 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783686 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783689 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783691 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783694 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.796079 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783697 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783700 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783708 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783711 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783714 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783716 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783720 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783723 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783726 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783729 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783731 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783736 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783739 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783742 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783744 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783747 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783749 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783752 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783754 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783757 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.796771 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783759 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783762 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783764 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783767 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783775 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783778 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783780 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783783 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783785 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783788 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783790 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783793 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783796 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783799 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783802 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783805 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783807 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783810 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783812 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.797484 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783814 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783817 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783820 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783824 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783831 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783834 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783853 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783857 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783863 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783866 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783869 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783871 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783874 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783877 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783881 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783884 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783888 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783891 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783901 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.798076 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783904 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783907 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783910 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783912 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783915 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783917 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783922 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783924 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783927 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783930 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783933 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783935 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783938 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783940 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783943 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783946 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783948 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783952 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783954 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783957 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.798565 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783959 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783962 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783964 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783971 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783974 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783977 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.783979 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.792083 2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.792102 2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.792122 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.792130 2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792237 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792245 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792250 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792255 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792260 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792264 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792268 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792273 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792277 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.799281 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792282 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792286 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792291 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792295 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792300 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792304 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792308 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792313 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792317 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792321 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792325 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792330 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792334 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792337 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792341 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792345 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792349 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792353 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792357 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792362 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.799809 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792367 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792372 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792376 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792381 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792387 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792392 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792396 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792403 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792408 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792413 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792418 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792423 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792427 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792432 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792436 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792443 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792448 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792452 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.800294 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792457 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792462 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792466 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792470 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792474 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792478 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792482 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792486 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792490 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792506 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792511 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792515 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792520 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792524 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792529 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792533 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792538 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792542 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792547 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792552 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.800759 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792556 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792560 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792564 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792569 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792573 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792577 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792581 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792586 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792590 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792596 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792600 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792604 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792608 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792612 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792616 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792620 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792625 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792629 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792633 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.801321 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792638 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792642 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792647 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792651 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792655 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792659 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792663 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792667 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792672 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792676 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792680 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792684 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792689 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792694 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792698 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792702 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792706 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792711 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792715 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.801751 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.792720 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.792761 2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793039 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793049 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793054 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793060 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793064 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793069 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793073 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793078 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793082 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793086 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793090 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793094 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793099 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793103 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.802258 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793116 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793120 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793125 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793129 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793133 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793138 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793142 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793146 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793151 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793155 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793159 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793165 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793169 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793173 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793177 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793182 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793186 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793190 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793194 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793198 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.802646 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793202 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793207 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793211 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793215 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793219 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793223 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793227 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793231 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793235 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793239 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793243 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793247 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793251 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793279 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793285 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793291 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793295 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793301 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793305 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793309 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.803161 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793313 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793318 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793322 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793326 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793331 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793336 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793340 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793347 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793352 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793356 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793361 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793365 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793369 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793374 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793378 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793383 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793387 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793391 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793395 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.803622 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793399 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793403 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793408 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793412 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793417 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793422 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793428 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793433 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793437 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793441 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793445 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793449 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793454 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793458 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793463 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793467 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793471 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793475 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.804077 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793480 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793484 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793488 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793492 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793496 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793501 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793505 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793510 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793514 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793518 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793523 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793527 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793531 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793536 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793540 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.793581 2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 22:32:41.804498 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793647 2574 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793653 2574 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793658 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793662 2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793667 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793671 2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793676 2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793680 2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793684 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793688 2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793692 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793696 2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793700 2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793704 2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793709 2574 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793713 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793717 2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793722 2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793726 2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793730 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:32:41.804943 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793735 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793739 2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793743 2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793748 2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793752 2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793756 2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793760 2574 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793765 2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793769 2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793774 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793782 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793786 2574 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793790 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793795 2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793799 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793803 2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793808 2574 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793812 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793816 2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793820 2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:32:41.805409 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793825 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793829 2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793849 2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793854 2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793858 2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793862 2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793869 2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793874 2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793879 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793883 2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793888 2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793892 2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793898 2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793903 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793907 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793911 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793916 2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793920 2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793924 2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:32:41.805903 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793928 2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793932 2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793936 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793942 2574 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793946 2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793952 2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793957 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793962 2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793967 2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793972 2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793977 2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793982 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793987 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793992 2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.793996 2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794002 2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794007 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794011 2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:32:41.806389 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794016 2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794020 2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794024 2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794028 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794032 2574 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794037 2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794041 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794047 2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794052 2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794057 2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794062 2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794066 2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794071 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794075 2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794080 2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794084 2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794088 2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794093 2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794097 2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794101 2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:32:41.806793 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794105 2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794110 2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794114 2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794118 2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794122 2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794126 2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794132 2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794137 2574 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:41.794141 2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.794216 2574 server.go:957] "Client rotation is on, will bootstrap in background" Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.799393 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.800203 2574 server.go:1014] "Starting client certificate rotation" Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.800293 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.800328 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.802078 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 22:32:41.807270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.803163 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 22:32:41.813326 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.813312 2574 log.go:25] "Validated CRI v1 runtime API" Oct 29 22:32:41.816871 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.816856 2574 log.go:25] "Validated CRI v1 image API" Oct 29 22:32:41.818047 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.818016 2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 29 22:32:41.820638 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.820619 2574 fs.go:135] Filesystem UUIDs: map[71ddfd82-bcc0-45fc-8bf2-2467e103f872:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f65ffe2d-f7db-4d59-86ed-2696b48b2ec8:/dev/nvme0n1p4] Oct 29 22:32:41.820715 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.820637 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 29 22:32:41.826118 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.826002 2574 manager.go:217] Machine: {Timestamp:2026-10-29 22:32:41.8246151 +0000 UTC m=+0.299960321 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102589 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec230f13f49fac39b0f1879e548ac801 SystemUUID:ec230f13-f49f-ac39-b0f1-879e548ac801 BootID:54af53cc-5e97-46bd-b7ce-40762eb8adfd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c8:08:ac:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c8:08:ac:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:c6:2b:a9:9c:62 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 29 22:32:41.826118 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.826112 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 29 22:32:41.826237 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.826210 2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 29 22:32:41.828961 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.828938 2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 29 22:32:41.829085 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.828962 2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-228.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 29 22:32:41.829135 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.829087 2574 topology_manager.go:143] "Creating topology manager with none policy" Oct 29 22:32:41.829135 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.829095 2574 container_manager_linux.go:311] "Creating device plugin manager" Oct 29 22:32:41.829191 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.829134 2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 22:32:41.829191 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.829152 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 22:32:41.830076 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.830065 2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 29 22:32:41.834118 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834107 2574 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 29 22:32:41.834239 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834229 2574 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 29 22:32:41.834880 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834870 2574 kubelet.go:493] "Attempting to sync node with API server" Oct 29 22:32:41.834917 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834888 2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 29 22:32:41.834917 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834907 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 29 22:32:41.834965 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834919 2574 kubelet.go:405] "Adding apiserver pod source" Oct 29 22:32:41.834965 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.834930 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 29 22:32:41.835454 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.835434 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 22:32:41.839337 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.839310 2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 29 22:32:41.845065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.845052 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 22:32:41.845877 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.845864 2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 29 22:32:41.845921 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.845896 2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 29 22:32:41.846065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846053 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 29 22:32:41.846093 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846072 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 29 22:32:41.846093 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846078 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 29 22:32:41.846093 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846084 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 29 22:32:41.846093 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846089 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 29 22:32:41.846093 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846094 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 29 22:32:41.846585 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846577 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 29 22:32:41.846612 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846587 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 29 22:32:41.846612 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846593 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 29 22:32:41.846612 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846599 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 29 22:32:41.846612 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846608 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 29 22:32:41.846732 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.846616 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 29 22:32:41.847204 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.847194 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 29 22:32:41.847233 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.847208 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 29 22:32:41.851481 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.851467 2574 server.go:1263] "Started kubelet" Oct 29 22:32:41.851570 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.851541 2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 29 22:32:41.851678 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.851627 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 29 22:32:41.851732 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.851708 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 29 22:32:41.852328 ip-10-0-136-228 systemd[1]: Started Kubernetes Kubelet. Oct 29 22:32:41.853643 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.853630 2574 server.go:317] "Adding debug handlers to kubelet server" Oct 29 22:32:41.856073 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.856053 2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 29 22:32:41.856141 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.856090 2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 29 22:32:41.860487 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.860465 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 29 22:32:41.861014 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.860994 2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 29 22:32:41.861198 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.861167 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:41.861754 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861736 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 29 22:32:41.861754 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861738 2574 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 29 22:32:41.861899 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861770 2574 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 29 22:32:41.861899 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861790 2574 factory.go:55] Registering systemd factory Oct 29 22:32:41.861899 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861801 2574 factory.go:223] Registration of the systemd container factory successfully Oct 29 22:32:41.861899 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861889 2574 reconstruct.go:97] "Volume reconstruction finished" Oct 29 22:32:41.861899 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.861899 2574 reconciler.go:29] "Reconciler: start to sync state" Oct 29 22:32:41.862065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.862006 2574 factory.go:153] Registering CRI-O factory Oct 29 22:32:41.862065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.862016 2574 factory.go:223] Registration of the crio container factory successfully Oct 29 22:32:41.862065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.862055 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 29 22:32:41.862179 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.862080 2574 factory.go:103] Registering Raw factory Oct 29 22:32:41.862179 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.862090 2574 manager.go:1199] Started watching for new ooms in manager Oct 29 22:32:41.863045 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.863010 2574 manager.go:319] Starting recovery of all containers Oct 29 22:32:41.874030 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.874013 2574 manager.go:324] Recovery completed Oct 29 22:32:41.875932 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.875915 2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 29 22:32:41.878014 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.877992 2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 29 22:32:41.878811 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.877993 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-228.ec2.internal.18e320d1bf3009d7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-228.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-228.ec2.internal,},FirstTimestamp:2026-10-29 22:32:41.851439575 +0000 UTC m=+0.326784799,LastTimestamp:2026-10-29 22:32:41.851439575 +0000 UTC m=+0.326784799,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-228.ec2.internal,}" Oct 29 22:32:41.879004 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.878992 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:41.880545 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.880529 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:41.880592 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.880562 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:41.880592 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.880574 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:41.881218 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881195 2574 cpu_manager.go:226] "Starting" policy="none" Oct 29 22:32:41.881218 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881206 2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 29 22:32:41.881314 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881220 2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 29 22:32:41.881314 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881245 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:41.881878 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881863 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:41.881939 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881892 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:41.881939 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.881905 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:41.883382 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.883370 2574 policy_none.go:50] "Start" Oct 29 22:32:41.883433 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.883384 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 29 22:32:41.883433 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.883394 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 29 22:32:41.885365 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.885355 2574 policy_none.go:44] "Start" Oct 29 22:32:41.888071 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.888053 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-228.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 29 22:32:41.898393 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.898374 2574 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 29 22:32:41.898468 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.898369 2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 29 22:32:41.898571 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.898513 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-228.ec2.internal.18e320d1c0ec2b46 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-228.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-228.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-228.ec2.internal,},FirstTimestamp:2026-10-29 22:32:41.880546118 +0000 UTC m=+0.355891348,LastTimestamp:2026-10-29 22:32:41.880546118 +0000 UTC m=+0.355891348,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-228.ec2.internal,}" Oct 29 22:32:41.923673 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.923642 2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 29 22:32:41.925447 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.925425 2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 29 22:32:41.925537 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.925459 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-228.ec2.internal.18e320d1c0ec81fb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-228.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-228.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-228.ec2.internal,},FirstTimestamp:2026-10-29 22:32:41.880568315 +0000 UTC m=+0.355913535,LastTimestamp:2026-10-29 22:32:41.880568315 +0000 UTC m=+0.355913535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-228.ec2.internal,}" Oct 29 22:32:41.925909 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.925896 2574 manager.go:341] "Starting Device Plugin manager" Oct 29 22:32:41.925979 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.925926 2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 29 22:32:41.925979 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.925939 2574 server.go:87] "Starting device plugin registration server" Oct 29 22:32:41.926174 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.926161 2574 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 29 22:32:41.926235 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.926174 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 29 22:32:41.926285 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.926275 2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 29 22:32:41.926360 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.926346 2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 29 22:32:41.926360 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.926358 2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 29 22:32:41.926927 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.926905 2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 29 22:32:41.927008 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.926956 2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:41.941438 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.941368 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-228.ec2.internal.18e320d1c0eca923 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-228.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-228.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-228.ec2.internal,},FirstTimestamp:2026-10-29 22:32:41.880578339 +0000 UTC m=+0.355923559,LastTimestamp:2026-10-29 22:32:41.880578339 +0000 UTC m=+0.355923559,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-228.ec2.internal,}" Oct 29 22:32:41.951677 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.951662 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l4kf2" Oct 29 22:32:41.959265 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:41.959194 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-228.ec2.internal.18e320d1c0ec2b46\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-228.ec2.internal.18e320d1c0ec2b46 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-228.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-228.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-228.ec2.internal,},FirstTimestamp:2026-10-29 22:32:41.880546118 +0000 UTC m=+0.355891348,LastTimestamp:2026-10-29 22:32:41.881878455 +0000 UTC m=+0.357223684,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-228.ec2.internal,}" Oct 29 22:32:41.979183 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:41.960050 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l4kf2" Oct 29 22:32:42.006032 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.005904 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 29 22:32:42.007307 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.007291 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 29 22:32:42.007387 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.007311 2574 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 29 22:32:42.007387 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.007329 2574 kubelet.go:2521] "Starting kubelet main sync loop" Oct 29 22:32:42.007387 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.007361 2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 29 22:32:42.023695 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.023676 2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:32:42.026786 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.026740 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:42.027447 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.027433 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:42.027501 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.027457 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:42.027501 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.027467 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:42.027501 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.027488 2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.046589 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.046574 2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.046635 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.046596 2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-228.ec2.internal\": node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.095829 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.095810 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.107601 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.107575 2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal"] Oct 29 22:32:42.107651 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.107631 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:42.108390 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.108371 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:42.108461 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.108396 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:42.108461 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.108405 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:42.110608 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.110595 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:42.110749 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.110734 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.110807 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.110769 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:42.111335 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.111313 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:42.111335 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.111334 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:42.111461 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.111344 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:42.111461 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.111371 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:42.111461 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.111388 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:42.111461 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.111396 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:42.113541 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.113528 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.113590 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.113551 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:42.114222 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.114205 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:42.114298 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.114231 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:42.114298 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.114240 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:42.144369 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.144341 2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-228.ec2.internal\" not found" node="ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.150643 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.150625 2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-228.ec2.internal\" not found" node="ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.163924 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.163908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.163975 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.163932 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e2359a413d1cf6872204355c65cbec7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" (UID: \"2e2359a413d1cf6872204355c65cbec7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.163975 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.163953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.196414 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.196398 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.264772 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.264753 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e2359a413d1cf6872204355c65cbec7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" (UID: \"2e2359a413d1cf6872204355c65cbec7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.264828 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.264784 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.264880 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.264864 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e2359a413d1cf6872204355c65cbec7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" (UID: \"2e2359a413d1cf6872204355c65cbec7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.264880 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.264870 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.264944 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.264861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.264944 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.264886 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.297069 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.297031 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.397607 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.397587 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.448178 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.448162 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.454521 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.454501 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:42.498197 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.498169 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.598733 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.598686 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.699171 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.699152 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.799707 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.799683 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.800091 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.799714 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 29 22:32:42.800091 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.799861 2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 29 22:32:42.900618 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:42.900555 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:42.943264 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:42.943236 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7d2aee7d03f12892682e2d7e593f68.slice/crio-75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7 WatchSource:0}: Error finding container 75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7: Status 404 returned error can't find the container with id 75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7 Oct 29 22:32:42.944011 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:42.943988 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2359a413d1cf6872204355c65cbec7.slice/crio-23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2 WatchSource:0}: Error finding container 23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2: Status 404 returned error can't find the container with id 23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2 Oct 29 22:32:42.948415 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.948398 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 22:32:42.965859 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.965801 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 22:27:42 +0000 UTC" deadline="2028-04-12 15:37:49.192318581 +0000 UTC" Oct 29 22:32:42.965859 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:42.965859 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12737h5m6.226462817s" Oct 29 22:32:43.001409 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:43.001388 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:43.010627 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.010585 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" event={"ID":"2e2359a413d1cf6872204355c65cbec7","Type":"ContainerStarted","Data":"23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2"} Oct 29 22:32:43.011519 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.011500 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" event={"ID":"ff7d2aee7d03f12892682e2d7e593f68","Type":"ContainerStarted","Data":"75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7"} Oct 29 22:32:43.051658 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.051641 2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:32:43.052719 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.052706 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:32:43.052776 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.052732 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:32:43.052776 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.052741 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:32:43.101677 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:43.101660 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:43.202319 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:43.202271 2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:32:43.209037 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.209021 2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:32:43.262145 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.262124 2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:32:43.273594 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.273576 2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:32:43.286582 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.286564 2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 22:32:43.288281 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.288268 2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:32:43.309055 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.309030 2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 22:32:43.342828 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.342808 2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:32:43.836169 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.836143 2574 apiserver.go:52] "Watching apiserver" Oct 29 22:32:43.861950 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.861925 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 29 22:32:43.879216 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.879196 2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 29 22:32:43.879548 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.879523 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-qtg9m","openshift-network-diagnostics/network-check-target-rh76g","kube-system/global-pull-secret-syncer-65b9p","kube-system/konnectivity-agent-skdsx","openshift-image-registry/node-ca-gkr9j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x","openshift-dns/node-resolver-9655m","openshift-multus/multus-hf4d4","openshift-ovn-kubernetes/ovnkube-node-prpm2","openshift-cluster-node-tuning-operator/tuned-wh7pt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal","openshift-multus/multus-additional-cni-plugins-wcwmt","openshift-network-operator/iptables-alerter-wc75t","kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal"] Oct 29 22:32:43.881822 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.881799 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:43.884119 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.884096 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:43.886201 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.886178 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:43.886311 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:43.886267 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:43.888457 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.888437 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:43.891196 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.890936 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:43.893060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.893042 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.895479 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.895307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899072 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899125 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bd868\"" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899184 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899258 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j7jzk\"" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899287 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899334 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.899374 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:43.899433 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.900744 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:43.900889 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:43.900798 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:43.902217 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.902202 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 29 22:32:43.903557 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.903166 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.907808 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.907773 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.909689 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.909115 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.910543 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.910516 2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 29 22:32:43.910644 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.910603 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2tjzr" Oct 29 22:32:43.963217 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.963194 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 29 22:32:43.974364 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974342 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-var-lib-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.974462 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974375 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-modprobe-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.974462 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-conf\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.974570 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974522 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-slash\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.974606 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974579 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-netd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.974658 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974608 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-system-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.974658 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-socket-dir-parent\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.974778 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974717 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-multus-certs\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.974834 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8wxt\" (UniqueName: \"kubernetes.io/projected/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-kube-api-access-g8wxt\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.974894 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974831 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/64f11246-8026-4e31-b461-1768145983a5-iptables-alerter-script\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:43.975038 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-device-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.975038 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-systemd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.975038 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.974968 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzfch\" (UniqueName: \"kubernetes.io/projected/19563c05-8911-48a9-bdf0-ca368119b49d-kube-api-access-zzfch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.975038 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975030 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-config\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.975237 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975088 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-host\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:43.975237 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975135 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:43.975237 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-tmp\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.975237 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975200 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-cnibin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.975237 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-cni-binary-copy\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.975448 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-kubelet-config\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:43.975448 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-netns\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.975448 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-serviceca\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:43.975448 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975374 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-557gg\" (UniqueName: \"kubernetes.io/projected/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-kube-api-access-557gg\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:43.975448 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-daemon-config\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.975659 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975450 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk8cd\" (UniqueName: \"kubernetes.io/projected/4fa778a3-926f-46e5-8a16-976b7f9819ca-kube-api-access-nk8cd\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.975659 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975479 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmq6n\" (UniqueName: \"kubernetes.io/projected/1ac5c5e2-a666-4cbf-9a73-677929dc640e-kube-api-access-nmq6n\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:43.975659 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975525 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19563c05-8911-48a9-bdf0-ca368119b49d-ovn-node-metrics-cert\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.975659 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-script-lib\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.975659 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975605 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-lib-modules\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.975659 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mrrj\" (UniqueName: \"kubernetes.io/projected/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-kube-api-access-7mrrj\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.975968 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975709 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-bin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.975968 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-hostroot\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.975968 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975806 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-netns\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.976131 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.975985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-ovn\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.976131 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976040 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-run\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.976131 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-kubelet\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.976131 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976123 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-agent-certs\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976172 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-system-cni-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976226 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-dbus\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976252 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-sys\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976279 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-multus\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cnibin\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.976344 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976335 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-sys-fs\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976376 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976412 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-k8s-cni-cncf-io\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-etc-kubernetes\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976470 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976497 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvg8\" (UniqueName: \"kubernetes.io/projected/64f11246-8026-4e31-b461-1768145983a5-kube-api-access-hwvg8\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976522 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-etc-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976560 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-systemd\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976618 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976642 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-log-socket\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.976693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-host\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976735 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-socket-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976760 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-kubelet\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976785 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw9pj\" (UniqueName: \"kubernetes.io/projected/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-kube-api-access-vw9pj\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976843 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysconfig\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976866 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-kubernetes\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976893 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976944 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.976985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac5c5e2-a666-4cbf-9a73-677929dc640e-hosts-file\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977035 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-systemd-units\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977061 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-konnectivity-ca\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977087 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-var-lib-kubelet\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977116 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac5c5e2-a666-4cbf-9a73-677929dc640e-tmp-dir\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:43.977261 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977182 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977213 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rx5d\" (UniqueName: \"kubernetes.io/projected/24348922-a2e2-4e39-9166-1aed0076055c-kube-api-access-7rx5d\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977242 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-node-log\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-env-overrides\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977288 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-tuned\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-conf-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f11246-8026-4e31-b461-1768145983a5-host-slash\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977368 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-registration-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977393 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-bin\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:43.978060 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:43.977419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:44.006912 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.006884 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 29 22:32:44.007069 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007051 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 29 22:32:44.007167 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007153 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 29 22:32:44.007213 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007185 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 29 22:32:44.007437 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007421 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 29 22:32:44.007437 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007434 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 29 22:32:44.007512 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007495 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 29 22:32:44.007548 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007539 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 29 22:32:44.007587 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007556 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 29 22:32:44.007587 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007581 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 29 22:32:44.007662 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007651 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-528cc\"" Oct 29 22:32:44.007763 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007747 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 29 22:32:44.007800 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007778 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 29 22:32:44.007870 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007859 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 29 22:32:44.007911 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007891 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 29 22:32:44.007970 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007959 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 29 22:32:44.008022 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007975 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:32:44.008022 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.007753 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 29 22:32:44.008113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.008052 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 29 22:32:44.008113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.008058 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jmngh\"" Oct 29 22:32:44.008113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.008075 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 29 22:32:44.008113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.008093 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 29 22:32:44.021434 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021412 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5fdr\"" Oct 29 22:32:44.021514 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021421 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qj89s\"" Oct 29 22:32:44.021640 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021618 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 29 22:32:44.021761 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021711 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-56msd\"" Oct 29 22:32:44.021761 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021719 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 29 22:32:44.021761 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021718 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pdbx4\"" Oct 29 22:32:44.021761 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.021747 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gqfpm\"" Oct 29 22:32:44.064333 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.064312 2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:32:44.078497 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078471 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-system-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.078617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078511 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-socket-dir-parent\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.078617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078538 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-multus-certs\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.078617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8wxt\" (UniqueName: \"kubernetes.io/projected/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-kube-api-access-g8wxt\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.078617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078590 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/64f11246-8026-4e31-b461-1768145983a5-iptables-alerter-script\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.078617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078613 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-device-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078637 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-systemd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzfch\" (UniqueName: \"kubernetes.io/projected/19563c05-8911-48a9-bdf0-ca368119b49d-kube-api-access-zzfch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-config\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078716 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-host\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-tmp\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-cnibin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078820 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-cni-binary-copy\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.078897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078874 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-kubelet-config\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-netns\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-serviceca\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078966 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-557gg\" (UniqueName: \"kubernetes.io/projected/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-kube-api-access-557gg\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.078991 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-daemon-config\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8cd\" (UniqueName: \"kubernetes.io/projected/4fa778a3-926f-46e5-8a16-976b7f9819ca-kube-api-access-nk8cd\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079040 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmq6n\" (UniqueName: \"kubernetes.io/projected/1ac5c5e2-a666-4cbf-9a73-677929dc640e-kube-api-access-nmq6n\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19563c05-8911-48a9-bdf0-ca368119b49d-ovn-node-metrics-cert\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079126 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-script-lib\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079147 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-lib-modules\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrrj\" (UniqueName: \"kubernetes.io/projected/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-kube-api-access-7mrrj\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079180 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-bin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079201 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-hostroot\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079224 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-netns\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079256 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-ovn\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-run\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-kubelet\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079380 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-agent-certs\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079410 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-system-cni-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079430 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079455 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-dbus\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079494 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-sys\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-multus\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cnibin\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-sys-fs\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079647 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-k8s-cni-cncf-io\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079724 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-etc-kubernetes\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079749 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.079781 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079770 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvg8\" (UniqueName: \"kubernetes.io/projected/64f11246-8026-4e31-b461-1768145983a5-kube-api-access-hwvg8\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079793 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-etc-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079819 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-systemd\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079851 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079877 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079908 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-log-socket\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-host\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079969 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.079995 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-socket-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080026 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-kubelet\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080080 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw9pj\" (UniqueName: \"kubernetes.io/projected/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-kube-api-access-vw9pj\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080106 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysconfig\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-kubernetes\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080137 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080182 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac5c5e2-a666-4cbf-9a73-677929dc640e-hosts-file\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.080351 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080223 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080255 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-systemd-units\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080280 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-konnectivity-ca\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080307 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-var-lib-kubelet\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac5c5e2-a666-4cbf-9a73-677929dc640e-tmp-dir\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080372 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080401 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rx5d\" (UniqueName: \"kubernetes.io/projected/24348922-a2e2-4e39-9166-1aed0076055c-kube-api-access-7rx5d\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-node-log\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080509 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-env-overrides\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-tuned\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-conf-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f11246-8026-4e31-b461-1768145983a5-host-slash\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-registration-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080656 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-bin\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080681 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-var-lib-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-modprobe-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080775 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-conf\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-slash\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080828 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-netd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.080956 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-netd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.081024 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-system-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.081150 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-socket-dir-parent\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.081851 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.081176 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-multus-certs\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.082206 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.081997 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/64f11246-8026-4e31-b461-1768145983a5-iptables-alerter-script\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.082293 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082257 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-etc-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.082404 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082333 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-systemd\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.082404 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082385 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.082531 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082425 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.082531 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082482 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-log-socket\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.082531 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082496 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.082686 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.082597 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:44.082686 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-socket-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.082686 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082661 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-kubelet\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.082686 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.082669 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret podName:3f37d2fc-3cf7-4105-9c73-bf532a71af91 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:44.582647789 +0000 UTC m=+3.057993019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret") pod "global-pull-secret-syncer-65b9p" (UID: "3f37d2fc-3cf7-4105-9c73-bf532a71af91") : object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:44.082922 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082726 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.082922 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-bin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.083046 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082910 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-hostroot\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.083046 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082974 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-device-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.083046 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083028 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-kubelet\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.083046 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083032 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-run\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.083246 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083120 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-kubernetes\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.083246 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083197 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.083359 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083250 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysconfig\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.083359 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083296 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-multus\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.083359 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083296 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.083359 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083337 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cnibin\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.083561 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083379 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-sys-fs\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.083561 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083420 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.083561 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083458 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-k8s-cni-cncf-io\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.083561 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083500 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-etc-kubernetes\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.083561 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083549 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-host\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.083855 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083832 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-config\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.083967 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083881 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.083967 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083896 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-sys\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.084071 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.083964 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-host\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.084071 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.082979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-netns\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.084176 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084076 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-env-overrides\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.084176 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084110 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-systemd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.084176 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084141 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-ovn\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.084334 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084235 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-system-cni-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.084334 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084251 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-dbus\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:44.084444 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084350 2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 29 22:32:44.084960 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.084932 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-cnibin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.085842 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.085181 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-lib-modules\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.085842 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.085745 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-kubelet-config\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:44.086594 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.086531 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:44.090376 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.090351 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-tmp\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.090479 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.090432 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-netns\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.090998 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.090975 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-agent-certs\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:44.092594 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.092569 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-cni-binary-copy\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.093145 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.093127 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-serviceca\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.093642 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.093573 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19563c05-8911-48a9-bdf0-ca368119b49d-ovn-node-metrics-cert\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.093730 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.093656 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:44.593626533 +0000 UTC m=+3.068971764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:44.093730 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.093721 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-conf-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.093848 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.093779 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f11246-8026-4e31-b461-1768145983a5-host-slash\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.093848 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.093839 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-script-lib\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.093972 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.093947 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-var-lib-kubelet\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.094043 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094022 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-registration-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.094130 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094108 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac5c5e2-a666-4cbf-9a73-677929dc640e-hosts-file\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.094191 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094173 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-systemd-units\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.094240 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094224 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-bin\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.094310 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094288 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-daemon-config\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.094422 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094401 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.094546 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094524 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-conf\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.094776 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094753 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-konnectivity-ca\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:44.094879 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.094804 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac5c5e2-a666-4cbf-9a73-677929dc640e-tmp-dir\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.095349 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.095321 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-var-lib-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.095487 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.095434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-modprobe-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.095535 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.095496 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-slash\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.095578 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.095563 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.095630 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.095616 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-node-log\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.096156 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.096129 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.097536 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.097509 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-tuned\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.103639 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.103620 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8wxt\" (UniqueName: \"kubernetes.io/projected/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-kube-api-access-g8wxt\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.106948 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.106927 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvg8\" (UniqueName: \"kubernetes.io/projected/64f11246-8026-4e31-b461-1768145983a5-kube-api-access-hwvg8\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.109298 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.109277 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 22:32:44.109429 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.109302 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 22:32:44.109429 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.109315 2574 projected.go:196] Error preparing data for projected volume kube-api-access-25p5w for pod openshift-network-diagnostics/network-check-target-rh76g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:44.109429 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.109381 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w podName:aa67ab99-04d4-4f83-a61b-d6f8fe0b0032 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:44.609363417 +0000 UTC m=+3.084708635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-25p5w" (UniqueName: "kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w") pod "network-check-target-rh76g" (UID: "aa67ab99-04d4-4f83-a61b-d6f8fe0b0032") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:44.111150 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.111126 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mrrj\" (UniqueName: \"kubernetes.io/projected/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-kube-api-access-7mrrj\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.111577 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.111550 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw9pj\" (UniqueName: \"kubernetes.io/projected/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-kube-api-access-vw9pj\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:44.112965 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.112911 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzfch\" (UniqueName: \"kubernetes.io/projected/19563c05-8911-48a9-bdf0-ca368119b49d-kube-api-access-zzfch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.113693 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.113627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmq6n\" (UniqueName: \"kubernetes.io/projected/1ac5c5e2-a666-4cbf-9a73-677929dc640e-kube-api-access-nmq6n\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.114232 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.114168 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8cd\" (UniqueName: \"kubernetes.io/projected/4fa778a3-926f-46e5-8a16-976b7f9819ca-kube-api-access-nk8cd\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.114985 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.114964 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-557gg\" (UniqueName: \"kubernetes.io/projected/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-kube-api-access-557gg\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.116820 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.116798 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rx5d\" (UniqueName: \"kubernetes.io/projected/24348922-a2e2-4e39-9166-1aed0076055c-kube-api-access-7rx5d\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.197031 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.197000 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9655m" Oct 29 22:32:44.209584 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.209552 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:32:44.210643 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.210612 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac5c5e2_a666_4cbf_9a73_677929dc640e.slice/crio-ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132 WatchSource:0}: Error finding container ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132: Status 404 returned error can't find the container with id ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132 Oct 29 22:32:44.217923 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.217896 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:44.221188 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.221149 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f11246_8026_4e31_b461_1768145983a5.slice/crio-65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2 WatchSource:0}: Error finding container 65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2: Status 404 returned error can't find the container with id 65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2 Oct 29 22:32:44.224011 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.223995 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:32:44.227866 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.227833 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5092efe_96fd_4aec_99f1_a2c07c84ac7f.slice/crio-d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905 WatchSource:0}: Error finding container d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905: Status 404 returned error can't find the container with id d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905 Oct 29 22:32:44.237214 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.237189 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:32:44.238000 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.237972 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7881bd2_9f24_4c58_9a77_68cc091ee9ab.slice/crio-39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91 WatchSource:0}: Error finding container 39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91: Status 404 returned error can't find the container with id 39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91 Oct 29 22:32:44.247845 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.247818 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24348922_a2e2_4e39_9166_1aed0076055c.slice/crio-19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b WatchSource:0}: Error finding container 19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b: Status 404 returned error can't find the container with id 19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b Oct 29 22:32:44.254185 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.254165 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:32:44.256809 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.256788 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hf4d4" Oct 29 22:32:44.263698 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.263676 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:32:44.265061 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.265035 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c8a747_4790_4ff6_9c11_a1b66d5a98c3.slice/crio-b91e3d71a2e3ef89e26951f90b3fe296dab75b256018dbceffd2bffb443c769f WatchSource:0}: Error finding container b91e3d71a2e3ef89e26951f90b3fe296dab75b256018dbceffd2bffb443c769f: Status 404 returned error can't find the container with id b91e3d71a2e3ef89e26951f90b3fe296dab75b256018dbceffd2bffb443c769f Oct 29 22:32:44.271721 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.270040 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa778a3_926f_46e5_8a16_976b7f9819ca.slice/crio-49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7 WatchSource:0}: Error finding container 49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7: Status 404 returned error can't find the container with id 49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7 Oct 29 22:32:44.279405 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.279182 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19563c05_8911_48a9_bdf0_ca368119b49d.slice/crio-507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d WatchSource:0}: Error finding container 507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d: Status 404 returned error can't find the container with id 507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d Oct 29 22:32:44.279405 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.279260 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:32:44.290688 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:32:44.290666 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ca05c6_f2bd_4ada_b0a1_b9cd07a89ff8.slice/crio-e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c WatchSource:0}: Error finding container e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c: Status 404 returned error can't find the container with id e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c Oct 29 22:32:44.584669 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.584643 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:44.584830 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.584809 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:44.584921 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.584898 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret podName:3f37d2fc-3cf7-4105-9c73-bf532a71af91 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:45.584876703 +0000 UTC m=+4.060221927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret") pod "global-pull-secret-syncer-65b9p" (UID: "3f37d2fc-3cf7-4105-9c73-bf532a71af91") : object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.685016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.685303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.685223 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.685409 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:45.685385438 +0000 UTC m=+4.160730666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.685497 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.685516 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.685530 2574 projected.go:196] Error preparing data for projected volume kube-api-access-25p5w for pod openshift-network-diagnostics/network-check-target-rh76g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:44.685603 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:44.685578 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w podName:aa67ab99-04d4-4f83-a61b-d6f8fe0b0032 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:45.685562374 +0000 UTC m=+4.160907586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-25p5w" (UniqueName: "kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w") pod "network-check-target-rh76g" (UID: "aa67ab99-04d4-4f83-a61b-d6f8fe0b0032") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:44.914504 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.914348 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 22:27:44 +0000 UTC" deadline="2028-07-11 23:14:42.485875982 +0000 UTC" Oct 29 22:32:44.914504 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:44.914415 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14904h41m57.571464837s" Oct 29 22:32:45.060299 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.060223 2574 generic.go:356] "Generic (PLEG): container finished" podID="ff7d2aee7d03f12892682e2d7e593f68" containerID="467428ddcd2b4b2f0450654c0b028a9438b3aab95a713986690471e1898c512d" exitCode=0 Oct 29 22:32:45.060712 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.060644 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" event={"ID":"ff7d2aee7d03f12892682e2d7e593f68","Type":"ContainerDied","Data":"467428ddcd2b4b2f0450654c0b028a9438b3aab95a713986690471e1898c512d"} Oct 29 22:32:45.074689 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.074611 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerStarted","Data":"e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c"} Oct 29 22:32:45.087293 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.087265 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d"} Oct 29 22:32:45.106788 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.106750 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" event={"ID":"28c8a747-4790-4ff6-9c11-a1b66d5a98c3","Type":"ContainerStarted","Data":"b91e3d71a2e3ef89e26951f90b3fe296dab75b256018dbceffd2bffb443c769f"} Oct 29 22:32:45.120868 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.120838 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gkr9j" event={"ID":"c7881bd2-9f24-4c58-9a77-68cc091ee9ab","Type":"ContainerStarted","Data":"39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91"} Oct 29 22:32:45.139818 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.139722 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9655m" event={"ID":"1ac5c5e2-a666-4cbf-9a73-677929dc640e","Type":"ContainerStarted","Data":"ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132"} Oct 29 22:32:45.144168 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.144138 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" event={"ID":"2e2359a413d1cf6872204355c65cbec7","Type":"ContainerStarted","Data":"2775dd6826ec442e90fe707cde3ce6a7b82235071831f5d4d476d8132944f15d"} Oct 29 22:32:45.167307 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.167008 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hf4d4" event={"ID":"4fa778a3-926f-46e5-8a16-976b7f9819ca","Type":"ContainerStarted","Data":"49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7"} Oct 29 22:32:45.171718 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.171692 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" event={"ID":"24348922-a2e2-4e39-9166-1aed0076055c","Type":"ContainerStarted","Data":"19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b"} Oct 29 22:32:45.175399 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.175374 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-skdsx" event={"ID":"e5092efe-96fd-4aec-99f1-a2c07c84ac7f","Type":"ContainerStarted","Data":"d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905"} Oct 29 22:32:45.179409 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.179384 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wc75t" event={"ID":"64f11246-8026-4e31-b461-1768145983a5","Type":"ContainerStarted","Data":"65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2"} Oct 29 22:32:45.593235 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.593155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:45.593411 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.593305 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:45.593411 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.593367 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret podName:3f37d2fc-3cf7-4105-9c73-bf532a71af91 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:47.593348743 +0000 UTC m=+6.068693959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret") pod "global-pull-secret-syncer-65b9p" (UID: "3f37d2fc-3cf7-4105-9c73-bf532a71af91") : object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:45.694538 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.694504 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:45.694700 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.694561 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:45.694839 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.694703 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 22:32:45.694839 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.694724 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 22:32:45.694839 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.694740 2574 projected.go:196] Error preparing data for projected volume kube-api-access-25p5w for pod openshift-network-diagnostics/network-check-target-rh76g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:45.694839 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.694799 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w podName:aa67ab99-04d4-4f83-a61b-d6f8fe0b0032 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:47.694781189 +0000 UTC m=+6.170126405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-25p5w" (UniqueName: "kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w") pod "network-check-target-rh76g" (UID: "aa67ab99-04d4-4f83-a61b-d6f8fe0b0032") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:45.695202 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.695183 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:45.695288 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:45.695243 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:47.695226836 +0000 UTC m=+6.170572051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:45.915617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.915507 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 22:27:44 +0000 UTC" deadline="2028-07-15 09:39:58.476690558 +0000 UTC" Oct 29 22:32:45.915617 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:45.915572 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14987h7m12.561123355s" Oct 29 22:32:46.009991 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:46.009962 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:46.010154 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:46.010099 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:46.010554 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:46.010533 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:46.010681 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:46.010660 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:46.010744 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:46.010729 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:46.010848 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:46.010800 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:46.186604 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:46.186499 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" event={"ID":"ff7d2aee7d03f12892682e2d7e593f68","Type":"ContainerStarted","Data":"1fa86ab569d8ee8b64d8cc429df5b4e001d034b86c6f14149fd5a5a2c5170a4f"} Oct 29 22:32:46.230263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:46.229717 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" podStartSLOduration=3.229703329 podStartE2EDuration="3.229703329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:46.228874814 +0000 UTC m=+4.704220049" watchObservedRunningTime="2026-10-29 22:32:46.229703329 +0000 UTC m=+4.705048562" Oct 29 22:32:46.230263 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:46.229992 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" podStartSLOduration=3.229985337 podStartE2EDuration="3.229985337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:45.190980637 +0000 UTC m=+3.666325871" watchObservedRunningTime="2026-10-29 22:32:46.229985337 +0000 UTC m=+4.705330572" Oct 29 22:32:47.202416 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:47.202377 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="4889c30a96688726f8d0559eaeaa09d0067ffabaa2d4f8d2804094288655d8fc" exitCode=0 Oct 29 22:32:47.202942 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:47.202501 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerDied","Data":"4889c30a96688726f8d0559eaeaa09d0067ffabaa2d4f8d2804094288655d8fc"} Oct 29 22:32:47.612158 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:47.612127 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:47.612351 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.612295 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:47.612420 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.612356 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret podName:3f37d2fc-3cf7-4105-9c73-bf532a71af91 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:51.612338505 +0000 UTC m=+10.087683720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret") pod "global-pull-secret-syncer-65b9p" (UID: "3f37d2fc-3cf7-4105-9c73-bf532a71af91") : object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:47.713288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:47.713326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.713429 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.713444 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.713454 2574 projected.go:196] Error preparing data for projected volume kube-api-access-25p5w for pod openshift-network-diagnostics/network-check-target-rh76g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.713499 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w podName:aa67ab99-04d4-4f83-a61b-d6f8fe0b0032 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:51.713485062 +0000 UTC m=+10.188830286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-25p5w" (UniqueName: "kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w") pod "network-check-target-rh76g" (UID: "aa67ab99-04d4-4f83-a61b-d6f8fe0b0032") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.713812 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:47.713870 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:47.713842 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:51.713833441 +0000 UTC m=+10.189178665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:48.011356 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:48.010419 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:48.011356 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:48.010549 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:48.011356 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:48.010989 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:48.011356 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:48.011103 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:48.011356 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:48.011166 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:48.011356 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:48.011243 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:49.214822 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:49.214742 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-skdsx" event={"ID":"e5092efe-96fd-4aec-99f1-a2c07c84ac7f","Type":"ContainerStarted","Data":"1a2cfd1b49a9809ab30507fb99fc8c1ea22722fd44e6f2bc0c7e4c644c676212"} Oct 29 22:32:50.010799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:50.010749 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:50.010985 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:50.010898 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:50.011078 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:50.010990 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:50.011137 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:50.011073 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:50.011137 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:50.011121 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:50.011234 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:50.011192 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:50.220718 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:50.220682 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gkr9j" event={"ID":"c7881bd2-9f24-4c58-9a77-68cc091ee9ab","Type":"ContainerStarted","Data":"dc5cc69817bb87580a14f7043fa6f9f9816ffcadbb7830ce5f89c7339ab489ec"} Oct 29 22:32:50.266415 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:50.265808 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-skdsx" podStartSLOduration=4.350736581 podStartE2EDuration="8.265792372s" totalImagesPullingTime="3.915055791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:49.270949368 +0000 UTC m=+7.746294602" watchObservedRunningTime="2026-10-29 22:32:50.265792372 +0000 UTC m=+8.741137601" Oct 29 22:32:50.266415 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:50.265908 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gkr9j" podStartSLOduration=3.283506908 podStartE2EDuration="8.265902271s" totalImagesPullingTime="4.982395363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:50.265354796 +0000 UTC m=+8.740700026" watchObservedRunningTime="2026-10-29 22:32:50.265902271 +0000 UTC m=+8.741247501" Oct 29 22:32:51.228278 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:51.228249 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" event={"ID":"24348922-a2e2-4e39-9166-1aed0076055c","Type":"ContainerStarted","Data":"9013e38fb196c59f31765dbcf9dc0cefc243f531f5c871e06ecf14405d0b58b2"} Oct 29 22:32:51.650889 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:51.650780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:51.651050 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.650994 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:51.651118 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.651057 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret podName:3f37d2fc-3cf7-4105-9c73-bf532a71af91 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:59.651039947 +0000 UTC m=+18.126385169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret") pod "global-pull-secret-syncer-65b9p" (UID: "3f37d2fc-3cf7-4105-9c73-bf532a71af91") : object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:51.751490 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:51.751296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:51.751647 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.751450 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:51.751647 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.751613 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:59.751594522 +0000 UTC m=+18.226939754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:51.752016 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:51.751993 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:51.752130 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.752117 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 22:32:51.752192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.752132 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 22:32:51.752192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.752141 2574 projected.go:196] Error preparing data for projected volume kube-api-access-25p5w for pod openshift-network-diagnostics/network-check-target-rh76g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:51.752283 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:51.752195 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w podName:aa67ab99-04d4-4f83-a61b-d6f8fe0b0032 nodeName:}" failed. No retries permitted until 2026-10-29 22:32:59.752183985 +0000 UTC m=+18.227529193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-25p5w" (UniqueName: "kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w") pod "network-check-target-rh76g" (UID: "aa67ab99-04d4-4f83-a61b-d6f8fe0b0032") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:52.008864 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.008768 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:52.008958 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:52.008874 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:52.008958 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.008953 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:52.009068 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:52.009027 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:52.009183 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.009164 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:52.009287 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:52.009257 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:52.046699 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.046671 2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 29 22:32:52.232753 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.232721 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" event={"ID":"24348922-a2e2-4e39-9166-1aed0076055c","Type":"ContainerStarted","Data":"4ea0dfa34f662cfc15352f56b557fbef9e68487f5b1553cf5ff3b2ea323f4705"} Oct 29 22:32:52.238196 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.235087 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerStarted","Data":"d5ab3323b943d6d0cf2d930ee3af615a8724deec55259f6b47cfe7949cc5fc3d"} Oct 29 22:32:52.425672 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.424013 2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:52.425672 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.424661 2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:32:52.932270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.931965 2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T22:32:52.04669374Z","UUID":"7d1c027b-3111-4758-b101-fbc642c2b609","Handler":null,"Name":"","Endpoint":""} Oct 29 22:32:52.933855 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.933812 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 29 22:32:52.933965 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:52.933861 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 29 22:32:53.240398 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.240316 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hf4d4" event={"ID":"4fa778a3-926f-46e5-8a16-976b7f9819ca","Type":"ContainerStarted","Data":"a9abcdcdaba2eabae7e49c7055c2b0295b6b34e567caddea837c21f390d0dc51"} Oct 29 22:32:53.243360 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.243287 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" event={"ID":"24348922-a2e2-4e39-9166-1aed0076055c","Type":"ContainerStarted","Data":"4698f8155640118430f959ce53ee1acc4cb0c7906c11dc2b5b205db69d063924"} Oct 29 22:32:53.245650 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.245622 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="d5ab3323b943d6d0cf2d930ee3af615a8724deec55259f6b47cfe7949cc5fc3d" exitCode=0 Oct 29 22:32:53.245943 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.245889 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerDied","Data":"d5ab3323b943d6d0cf2d930ee3af615a8724deec55259f6b47cfe7949cc5fc3d"} Oct 29 22:32:53.249212 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.249195 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9655m" event={"ID":"1ac5c5e2-a666-4cbf-9a73-677929dc640e","Type":"ContainerStarted","Data":"f0bea36745a441313f0ec6438fb3bb0766b2e5dcfa0a48de8f5cf24b50ca8374"} Oct 29 22:32:53.282823 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.282770 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hf4d4" podStartSLOduration=3.202512376 podStartE2EDuration="11.282753816s" totalImagesPullingTime="8.08024144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:53.282261155 +0000 UTC m=+11.757606387" watchObservedRunningTime="2026-10-29 22:32:53.282753816 +0000 UTC m=+11.758099051" Oct 29 22:32:53.365291 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:53.365225 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-9655m" podStartSLOduration=3.126199413 podStartE2EDuration="11.365206376s" totalImagesPullingTime="8.239006963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:53.364470874 +0000 UTC m=+11.839816104" watchObservedRunningTime="2026-10-29 22:32:53.365206376 +0000 UTC m=+11.840551607" Oct 29 22:32:54.009714 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.007869 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:54.009714 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.007870 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:54.009714 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:54.007978 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:54.009714 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.008002 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:54.009714 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:54.008065 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:54.009714 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:54.008145 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:54.252759 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.252722 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wc75t" event={"ID":"64f11246-8026-4e31-b461-1768145983a5","Type":"ContainerStarted","Data":"6a763331eba2cd4acb4393aaecd315b642c030ba0c5dcc0254fd677fcefebf1a"} Oct 29 22:32:54.256091 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.256062 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="e0f0ce8ddb2c67bd27ba912d1ac89307e96a9779625905e823cbfa002fb52a79" exitCode=0 Oct 29 22:32:54.256293 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.256245 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerDied","Data":"e0f0ce8ddb2c67bd27ba912d1ac89307e96a9779625905e823cbfa002fb52a79"} Oct 29 22:32:54.293032 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.292961 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" podStartSLOduration=3.9312792979999998 podStartE2EDuration="12.292947933s" totalImagesPullingTime="8.361668635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:53.407327865 +0000 UTC m=+11.882673096" watchObservedRunningTime="2026-10-29 22:32:54.292947933 +0000 UTC m=+12.768293171" Oct 29 22:32:54.338581 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:54.338526 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wc75t" podStartSLOduration=4.367928371 podStartE2EDuration="12.338508835s" totalImagesPullingTime="7.970580464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:54.292776073 +0000 UTC m=+12.768121303" watchObservedRunningTime="2026-10-29 22:32:54.338508835 +0000 UTC m=+12.813854067" Oct 29 22:32:55.261105 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:55.261024 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="df2a14be32fa37a4210d9d51000ddbbc9268d98d9a671178bd22642500079fdd" exitCode=0 Oct 29 22:32:55.261725 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:55.261117 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerDied","Data":"df2a14be32fa37a4210d9d51000ddbbc9268d98d9a671178bd22642500079fdd"} Oct 29 22:32:55.263045 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:55.263020 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" event={"ID":"28c8a747-4790-4ff6-9c11-a1b66d5a98c3","Type":"ContainerStarted","Data":"bc9d81c99a050071fb44cc64ca530261a6f5db0ee9bfec8ae23b1d7acd8f9917"} Oct 29 22:32:55.343824 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:55.343780 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" podStartSLOduration=3.001046314 podStartE2EDuration="13.343766837s" totalImagesPullingTime="10.342720523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:32:55.343757559 +0000 UTC m=+13.819102788" watchObservedRunningTime="2026-10-29 22:32:55.343766837 +0000 UTC m=+13.819112067" Oct 29 22:32:56.008059 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:56.008027 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:56.008059 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:56.008043 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:56.008352 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:56.008152 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:56.008352 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:56.008198 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:56.008352 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:56.008284 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:56.008488 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:56.008383 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:58.011707 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:58.011481 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:58.012379 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:58.011513 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:58.012379 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:58.011813 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:32:58.012379 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:58.011903 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:32:58.012379 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:58.011524 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:58.012379 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:58.011988 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:32:58.272499 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:58.272425 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="8d3c9dc4f280f416caf642a7cba3c0a35a80ffdf98ad8c0639bee720b748c064" exitCode=0 Oct 29 22:32:58.272662 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:58.272491 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerDied","Data":"8d3c9dc4f280f416caf642a7cba3c0a35a80ffdf98ad8c0639bee720b748c064"} Oct 29 22:32:59.278777 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:59.278755 2574 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="82be52e091ce1b3dc712d38781d22b4d78538ff806e6d548b0aa6ae4d6a0cc4e" exitCode=0 Oct 29 22:32:59.279150 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:59.278860 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerDied","Data":"82be52e091ce1b3dc712d38781d22b4d78538ff806e6d548b0aa6ae4d6a0cc4e"} Oct 29 22:32:59.280945 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:59.280926 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"42a758f41155ec1586815d13b0300b4b0022efe9904f7d490d1b5cb5a6bc81fa"} Oct 29 22:32:59.715559 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:59.715349 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:32:59.715559 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.715482 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:59.715722 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.715611 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret podName:3f37d2fc-3cf7-4105-9c73-bf532a71af91 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.715597177 +0000 UTC m=+34.190942398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret") pod "global-pull-secret-syncer-65b9p" (UID: "3f37d2fc-3cf7-4105-9c73-bf532a71af91") : object "kube-system"/"original-pull-secret" not registered Oct 29 22:32:59.816008 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:59.815967 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:32:59.816008 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:32:59.816014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:32:59.816192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.816104 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:59.816192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.816118 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 22:32:59.816192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.816136 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 22:32:59.816192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.816147 2574 projected.go:196] Error preparing data for projected volume kube-api-access-25p5w for pod openshift-network-diagnostics/network-check-target-rh76g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:32:59.816192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.816167 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.816148514 +0000 UTC m=+34.291493727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 22:32:59.816192 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:32:59.816183 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w podName:aa67ab99-04d4-4f83-a61b-d6f8fe0b0032 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.816173558 +0000 UTC m=+34.291518784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-25p5w" (UniqueName: "kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w") pod "network-check-target-rh76g" (UID: "aa67ab99-04d4-4f83-a61b-d6f8fe0b0032") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 22:33:00.008420 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.008359 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:00.008420 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.008373 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:00.008572 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:00.008458 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:33:00.008572 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.008503 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:00.008631 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:00.008570 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:33:00.008631 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:00.008622 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:33:00.286165 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.286083 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" event={"ID":"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8","Type":"ContainerStarted","Data":"918fa91be0d9bed5a5d06b34f01e68cdec09df261a6fa23d277fb5dca18e1f62"} Oct 29 22:33:00.288970 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.288948 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"af75487ceb55b12374eb679be7d15a56567698df962e81f5bc0c22177a8fdcc6"} Oct 29 22:33:00.289072 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.289007 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"b4618426145a9d19ae05636c7bc720920fa23dcd35d64a9a08a67a53555d86da"} Oct 29 22:33:00.289072 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.289045 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"205e3f6994ac67dc755d1d0f1c93c1085a948a7b648626de4bb0e0932010b819"} Oct 29 22:33:00.289143 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.289074 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"764059e05dab5056f9b8255b1337e6a5d442f4aeec783403c5a141384020dded"} Oct 29 22:33:00.289143 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.289108 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"a53fa443a94a7f5fd8212ecf7d894fb2aaae50b9bf3ac052b8b99dbdb6907004"} Oct 29 22:33:00.332547 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:00.332506 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wcwmt" podStartSLOduration=7.826516602 podStartE2EDuration="18.332496407s" totalImagesPullingTime="10.505979805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:00.332086928 +0000 UTC m=+18.807432158" watchObservedRunningTime="2026-10-29 22:33:00.332496407 +0000 UTC m=+18.807841636" Oct 29 22:33:01.230580 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:01.230551 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:01.230755 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:01.230672 2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 22:33:01.231364 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:01.231335 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:02.010718 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:02.010654 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:02.011144 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:02.010768 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:02.011144 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:02.010777 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:33:02.011144 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:02.010859 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:33:02.011144 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:02.010897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:02.011144 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:02.010975 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:33:02.296103 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:02.296031 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"4cc75759d1c8af3994c2caffa072ba11822fbbd35bcd5a3280fb4cbf499228e4"} Oct 29 22:33:04.008204 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.008133 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:04.008957 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.008141 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:04.008957 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:04.008229 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:33:04.008957 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:04.008311 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:33:04.008957 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.008344 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:04.008957 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:04.008396 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:33:04.304150 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.304073 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" event={"ID":"19563c05-8911-48a9-bdf0-ca368119b49d","Type":"ContainerStarted","Data":"9b98671f459f0e4b6f71020305b0dc80fdcdd97484b02bd4f99e4169c2ccd8f1"} Oct 29 22:33:04.304352 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.304302 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:04.320160 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.320139 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:04.394913 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:04.394871 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" podStartSLOduration=7.510579847 podStartE2EDuration="22.394860981s" totalImagesPullingTime="14.884281134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:04.381294542 +0000 UTC m=+22.856639778" watchObservedRunningTime="2026-10-29 22:33:04.394860981 +0000 UTC m=+22.870206221" Oct 29 22:33:05.104440 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.104268 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:05.306985 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.306949 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:05.324348 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.324321 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:05.726804 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.726776 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-65b9p"] Oct 29 22:33:05.726954 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.726902 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:05.727005 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:05.726987 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:33:05.730441 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.730411 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qtg9m"] Oct 29 22:33:05.730570 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.730539 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:05.730645 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:05.730628 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:33:05.730971 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.730952 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rh76g"] Oct 29 22:33:05.731050 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:05.731040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:05.731124 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:05.731108 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:33:08.008181 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:08.008153 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:08.008181 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:08.008164 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:08.008652 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:08.008189 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:08.008652 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:08.008304 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:33:08.008652 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:08.008480 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:33:08.008652 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:08.008575 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:33:10.008118 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:10.008085 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:10.008551 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:10.008141 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:10.008551 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:10.008145 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:10.008551 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:10.008237 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qtg9m" podUID="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" Oct 29 22:33:10.008551 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:10.008340 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65b9p" podUID="3f37d2fc-3cf7-4105-9c73-bf532a71af91" Oct 29 22:33:10.008551 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:10.008392 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh76g" podUID="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" Oct 29 22:33:11.207153 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.207076 2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeReady" Oct 29 22:33:11.207639 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.207198 2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 29 22:33:11.253258 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.253228 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-jlpb2"] Oct 29 22:33:11.255733 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.255708 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:11.256783 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.256763 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-6z7h7"] Oct 29 22:33:11.259420 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.259403 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d"] Oct 29 22:33:11.259566 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.259547 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:11.261973 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.261956 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xtkwb"] Oct 29 22:33:11.262146 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.262132 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:11.264632 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.264615 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-m855q"] Oct 29 22:33:11.264728 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.264712 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:11.267256 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.267238 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t"] Oct 29 22:33:11.267338 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.267304 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.269979 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.269962 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66bf65fd8-phdn2"] Oct 29 22:33:11.270088 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.270068 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.274020 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.273999 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p"] Oct 29 22:33:11.278270 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.278247 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.280107 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.280082 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.280199 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.280130 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.280199 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.280130 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d"] Oct 29 22:33:11.280313 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.280214 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.289185 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.283998 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fdff46b97-drfgt"] Oct 29 22:33:11.289185 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.284535 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.289185 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.284855 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j5zfb\"" Oct 29 22:33:11.289486 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.289467 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 29 22:33:11.291252 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.291234 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8p2vc"] Oct 29 22:33:11.291354 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.291346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.294872 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.294783 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-w564k"] Oct 29 22:33:11.295245 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.295104 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.302294 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.302261 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp"] Oct 29 22:33:11.302962 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.302941 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:11.306206 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.306187 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b6bb75897-sr2f4"] Oct 29 22:33:11.306301 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.306295 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.309805 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.309785 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk"] Oct 29 22:33:11.310922 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.310906 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.312794 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.312683 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vml7h"] Oct 29 22:33:11.313997 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.313977 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:11.316460 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.316443 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rrr8p"] Oct 29 22:33:11.316580 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.316562 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.320514 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320476 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-jlpb2"] Oct 29 22:33:11.320689 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320667 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xtkwb"] Oct 29 22:33:11.320789 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320710 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d"] Oct 29 22:33:11.320789 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320732 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-6z7h7"] Oct 29 22:33:11.320789 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320754 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t"] Oct 29 22:33:11.320789 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320778 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-m855q"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320797 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-w564k"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320815 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320833 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320868 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrr8p"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320887 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320912 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fdff46b97-drfgt"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320952 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.320974 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8p2vc"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.321003 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vml7h"] Oct 29 22:33:11.321034 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.321028 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66bf65fd8-phdn2"] Oct 29 22:33:11.321508 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.321046 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b6bb75897-sr2f4"] Oct 29 22:33:11.321508 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.321118 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.375249 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375229 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.375373 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375359 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.375373 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375359 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.375491 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375477 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.375552 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375507 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.375552 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375542 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.375666 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375568 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.375666 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375656 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.375799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375691 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.375799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375712 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.375799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375712 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.375799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375729 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 29 22:33:11.375799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375726 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375920 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375958 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375971 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375986 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375986 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376003 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.376113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.375987 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.376484 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376156 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.376484 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376167 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 29 22:33:11.376484 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376173 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.376484 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376157 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 29 22:33:11.376484 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376218 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 29 22:33:11.376484 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376449 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 29 22:33:11.376817 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376567 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 29 22:33:11.376817 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.376645 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 29 22:33:11.390197 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.390176 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 29 22:33:11.390197 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.390179 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 29 22:33:11.390197 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.390206 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 29 22:33:11.390197 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.390205 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 29 22:33:11.390473 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.390457 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 29 22:33:11.390558 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.390501 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 29 22:33:11.402298 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402281 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 29 22:33:11.402365 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402298 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 29 22:33:11.402365 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402329 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 29 22:33:11.402365 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402342 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 29 22:33:11.402365 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402349 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 29 22:33:11.402849 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402823 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 29 22:33:11.402886 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.402852 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 29 22:33:11.403103 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403089 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xx9bd\"" Oct 29 22:33:11.403332 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403321 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 29 22:33:11.403368 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403355 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j2wnd\"" Oct 29 22:33:11.403685 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403646 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ggjqq\"" Oct 29 22:33:11.403685 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403649 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9vstz\"" Oct 29 22:33:11.403685 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403683 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-6bbfq\"" Oct 29 22:33:11.403801 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403688 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 29 22:33:11.403801 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.403680 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-dpwt7\"" Oct 29 22:33:11.411380 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411362 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8b45469-5973-49bc-b8c2-8ec0dce0e242-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.411463 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411390 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.411463 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411409 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8db65a63-2853-43db-8877-83a0d26c2de2-cert\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.411463 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411425 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.411547 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ghgf\" (UniqueName: \"kubernetes.io/projected/230e7bac-638f-4478-989a-abad8f7213bd-kube-api-access-4ghgf\") pod \"kas-connection-checker-685895c7c9-6z7h7\" (UID: \"230e7bac-638f-4478-989a-abad8f7213bd\") " pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:11.411547 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-serving-cert\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.411605 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqblx\" (UniqueName: \"kubernetes.io/projected/8db65a63-2853-43db-8877-83a0d26c2de2-kube-api-access-wqblx\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.411605 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411569 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjjq\" (UniqueName: \"kubernetes.io/projected/983a2044-473f-48be-8e73-5da2ed82aaf4-kube-api-access-bpjjq\") pod \"kas-connection-checker-685895c7c9-jlpb2\" (UID: \"983a2044-473f-48be-8e73-5da2ed82aaf4\") " pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:11.411605 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bzv6\" (UniqueName: \"kubernetes.io/projected/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-kube-api-access-6bzv6\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.411689 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411634 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.411689 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411667 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnxbw\" (UniqueName: \"kubernetes.io/projected/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-kube-api-access-gnxbw\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.411689 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-default-certificate\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.411815 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411733 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.411815 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411752 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb729\" (UniqueName: \"kubernetes.io/projected/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-kube-api-access-vb729\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.411815 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411770 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.411815 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkht6\" (UniqueName: \"kubernetes.io/projected/5332e2aa-b433-47d6-b3c3-d42e66376eca-kube-api-access-mkht6\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.411947 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411833 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa7fc2d-d350-4aba-977b-32868a6e3fe7-tmp-dir\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.411947 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.412004 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411952 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412004 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411975 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.412004 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.411992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412088 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412008 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-tmp\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.412088 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412029 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.412141 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwxrx\" (UniqueName: \"kubernetes.io/projected/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9-kube-api-access-zwxrx\") pod \"volume-data-source-validator-58c9f5f46d-829vk\" (UID: \"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:11.412141 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412105 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412141 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412119 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.412224 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-trusted-ca\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.412224 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412176 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-config\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.412224 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.412224 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpmqc\" (UniqueName: \"kubernetes.io/projected/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20-kube-api-access-zpmqc\") pod \"kas-connection-checker-685895c7c9-xtkwb\" (UID: \"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20\") " pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:11.412334 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412334 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.412334 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412291 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:11.412410 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412365 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.412410 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412394 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412468 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412416 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q87h4\" (UniqueName: \"kubernetes.io/projected/d8b45469-5973-49bc-b8c2-8ec0dce0e242-kube-api-access-q87h4\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.412468 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412433 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw8wf\" (UniqueName: \"kubernetes.io/projected/4ec29132-5b42-4a06-be85-de3789f00145-kube-api-access-lw8wf\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.412468 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-service-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.412556 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412487 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3a898c-87c6-409d-a70f-f01e8e15d787-serving-cert\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.412556 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khhs6\" (UniqueName: \"kubernetes.io/projected/ed3a898c-87c6-409d-a70f-f01e8e15d787-kube-api-access-khhs6\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.412556 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bba55700-7326-482d-919f-02f3eff316a1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:11.412640 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.412640 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412621 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.412696 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412648 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t79s\" (UniqueName: \"kubernetes.io/projected/efa7fc2d-d350-4aba-977b-32868a6e3fe7-kube-api-access-8t79s\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.412696 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-snapshots\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.412752 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412705 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hkth\" (UniqueName: \"kubernetes.io/projected/24316779-cb78-42df-954b-c36d2b0e097d-kube-api-access-8hkth\") pod \"network-check-source-6f78d59f99-mms8d\" (UID: \"24316779-cb78-42df-954b-c36d2b0e097d\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:11.412752 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412733 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.412752 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412748 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efa7fc2d-d350-4aba-977b-32868a6e3fe7-config-volume\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.412834 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412769 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-stats-auth\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.412834 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412804 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.412834 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412829 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.412935 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412876 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412935 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412893 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.412935 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b45469-5973-49bc-b8c2-8ec0dce0e242-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.413014 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ec29132-5b42-4a06-be85-de3789f00145-config\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.413014 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.412980 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ec29132-5b42-4a06-be85-de3789f00145-serving-cert\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.417833 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.417817 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 29 22:33:11.417897 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.417821 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 29 22:33:11.417973 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.417958 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 29 22:33:11.513564 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513545 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.513709 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.513709 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513588 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.513709 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b45469-5973-49bc-b8c2-8ec0dce0e242-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.513709 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513639 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ec29132-5b42-4a06-be85-de3789f00145-config\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.513709 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.513694 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:11.513709 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.513710 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bf65fd8-phdn2: secret "image-registry-tls" not found Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.513744 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.513760 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls podName:eea9c729-9323-4535-bb3b-f5e67bb28b29 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.013741664 +0000 UTC m=+30.489086887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls") pod "image-registry-66bf65fd8-phdn2" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29") : secret "image-registry-tls" not found Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513778 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ec29132-5b42-4a06-be85-de3789f00145-serving-cert\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.513799 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls podName:5332e2aa-b433-47d6-b3c3-d42e66376eca nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.013783015 +0000 UTC m=+30.489128249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-gwr5t" (UID: "5332e2aa-b433-47d6-b3c3-d42e66376eca") : secret "samples-operator-tls" not found Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513870 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8b45469-5973-49bc-b8c2-8ec0dce0e242-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513903 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513930 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8db65a63-2853-43db-8877-83a0d26c2de2-cert\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513957 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.513982 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ghgf\" (UniqueName: \"kubernetes.io/projected/230e7bac-638f-4478-989a-abad8f7213bd-kube-api-access-4ghgf\") pod \"kas-connection-checker-685895c7c9-6z7h7\" (UID: \"230e7bac-638f-4478-989a-abad8f7213bd\") " pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514006 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-serving-cert\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.514044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514031 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqblx\" (UniqueName: \"kubernetes.io/projected/8db65a63-2853-43db-8877-83a0d26c2de2-kube-api-access-wqblx\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514054 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjjq\" (UniqueName: \"kubernetes.io/projected/983a2044-473f-48be-8e73-5da2ed82aaf4-kube-api-access-bpjjq\") pod \"kas-connection-checker-685895c7c9-jlpb2\" (UID: \"983a2044-473f-48be-8e73-5da2ed82aaf4\") " pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bzv6\" (UniqueName: \"kubernetes.io/projected/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-kube-api-access-6bzv6\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514107 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514129 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnxbw\" (UniqueName: \"kubernetes.io/projected/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-kube-api-access-gnxbw\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-default-certificate\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514193 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb729\" (UniqueName: \"kubernetes.io/projected/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-kube-api-access-vb729\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514253 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ec29132-5b42-4a06-be85-de3789f00145-config\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkht6\" (UniqueName: \"kubernetes.io/projected/5332e2aa-b433-47d6-b3c3-d42e66376eca-kube-api-access-mkht6\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa7fc2d-d350-4aba-977b-32868a6e3fe7-tmp-dir\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514329 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514389 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.514569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-tmp\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514473 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514505 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwxrx\" (UniqueName: \"kubernetes.io/projected/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9-kube-api-access-zwxrx\") pod \"volume-data-source-validator-58c9f5f46d-829vk\" (UID: \"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514582 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-trusted-ca\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514610 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-config\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514663 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpmqc\" (UniqueName: \"kubernetes.io/projected/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20-kube-api-access-zpmqc\") pod \"kas-connection-checker-685895c7c9-xtkwb\" (UID: \"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20\") " pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514689 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514717 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514808 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514852 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87h4\" (UniqueName: \"kubernetes.io/projected/d8b45469-5973-49bc-b8c2-8ec0dce0e242-kube-api-access-q87h4\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.515375 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw8wf\" (UniqueName: \"kubernetes.io/projected/4ec29132-5b42-4a06-be85-de3789f00145-kube-api-access-lw8wf\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-service-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3a898c-87c6-409d-a70f-f01e8e15d787-serving-cert\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514960 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khhs6\" (UniqueName: \"kubernetes.io/projected/ed3a898c-87c6-409d-a70f-f01e8e15d787-kube-api-access-khhs6\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.514974 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bba55700-7326-482d-919f-02f3eff316a1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.515026 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.015012553 +0000 UTC m=+30.490357775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514238 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515045 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515161 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t79s\" (UniqueName: \"kubernetes.io/projected/efa7fc2d-d350-4aba-977b-32868a6e3fe7-kube-api-access-8t79s\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515187 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-snapshots\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hkth\" (UniqueName: \"kubernetes.io/projected/24316779-cb78-42df-954b-c36d2b0e097d-kube-api-access-8hkth\") pod \"network-check-source-6f78d59f99-mms8d\" (UID: \"24316779-cb78-42df-954b-c36d2b0e097d\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515271 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efa7fc2d-d350-4aba-977b-32868a6e3fe7-config-volume\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515279 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8b45469-5973-49bc-b8c2-8ec0dce0e242-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.516094 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-stats-auth\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515685 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bba55700-7326-482d-919f-02f3eff316a1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515753 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-tmp\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.515903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.515992 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.01597783 +0000 UTC m=+30.491323057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : configmap references non-existent config key: service-ca.crt Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.516146 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.516193 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls podName:2ce7d6cc-47fa-4a6a-ba52-5a23efd31019 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.01617865 +0000 UTC m=+30.491523883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lmjqp" (UID: "2ce7d6cc-47fa-4a6a-ba52-5a23efd31019") : secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.516250 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.516283 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert podName:bba55700-7326-482d-919f-02f3eff316a1 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.016271083 +0000 UTC m=+30.491616293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-w564k" (UID: "bba55700-7326-482d-919f-02f3eff316a1") : secret "networking-console-plugin-cert" not found Oct 29 22:33:11.516875 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.516363 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.517389 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.516904 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.517389 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.516969 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.517389 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.517059 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-snapshots\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.519510 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.517675 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.519510 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.517961 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.519510 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.514903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.519510 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.518804 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efa7fc2d-d350-4aba-977b-32868a6e3fe7-config-volume\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.519510 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519150 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa7fc2d-d350-4aba-977b-32868a6e3fe7-tmp-dir\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.519510 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519226 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-trusted-ca\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.519898 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519611 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b45469-5973-49bc-b8c2-8ec0dce0e242-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.519898 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519767 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-config\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.519898 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519875 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8db65a63-2853-43db-8877-83a0d26c2de2-cert\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.520044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519935 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-serving-cert\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.520044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519964 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ec29132-5b42-4a06-be85-de3789f00145-serving-cert\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.520044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.519965 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-default-certificate\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.520193 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.520044 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:11.520193 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.520095 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.020079952 +0000 UTC m=+30.495425183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:11.520317 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.520291 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-service-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.520499 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.520408 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:11.520499 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.520425 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fdff46b97-drfgt: secret "image-registry-tls" not found Oct 29 22:33:11.520499 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.520459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.520499 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:11.520479 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls podName:05dc76a9-2747-4bd0-907d-a9b2caa3b25d nodeName:}" failed. No retries permitted until 2026-10-29 22:33:12.020463567 +0000 UTC m=+30.495808792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls") pod "image-registry-7fdff46b97-drfgt" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d") : secret "image-registry-tls" not found Oct 29 22:33:11.520864 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.520817 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.520972 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.520817 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.521585 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.521562 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.521713 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.521698 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3a898c-87c6-409d-a70f-f01e8e15d787-serving-cert\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.522242 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.522227 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-stats-auth\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.522714 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.522692 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.535861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.535818 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ghgf\" (UniqueName: \"kubernetes.io/projected/230e7bac-638f-4478-989a-abad8f7213bd-kube-api-access-4ghgf\") pod \"kas-connection-checker-685895c7c9-6z7h7\" (UID: \"230e7bac-638f-4478-989a-abad8f7213bd\") " pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:11.536321 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.536295 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqblx\" (UniqueName: \"kubernetes.io/projected/8db65a63-2853-43db-8877-83a0d26c2de2-kube-api-access-wqblx\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.538055 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.537993 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnxbw\" (UniqueName: \"kubernetes.io/projected/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-kube-api-access-gnxbw\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.538143 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.538100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjjq\" (UniqueName: \"kubernetes.io/projected/983a2044-473f-48be-8e73-5da2ed82aaf4-kube-api-access-bpjjq\") pod \"kas-connection-checker-685895c7c9-jlpb2\" (UID: \"983a2044-473f-48be-8e73-5da2ed82aaf4\") " pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:11.540292 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.539698 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.540292 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.539859 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkht6\" (UniqueName: \"kubernetes.io/projected/5332e2aa-b433-47d6-b3c3-d42e66376eca-kube-api-access-mkht6\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:11.540489 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.540468 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t79s\" (UniqueName: \"kubernetes.io/projected/efa7fc2d-d350-4aba-977b-32868a6e3fe7-kube-api-access-8t79s\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:11.541340 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.541318 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwxrx\" (UniqueName: \"kubernetes.io/projected/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9-kube-api-access-zwxrx\") pod \"volume-data-source-validator-58c9f5f46d-829vk\" (UID: \"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:11.542672 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.542402 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khhs6\" (UniqueName: \"kubernetes.io/projected/ed3a898c-87c6-409d-a70f-f01e8e15d787-kube-api-access-khhs6\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.542672 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.542603 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87h4\" (UniqueName: \"kubernetes.io/projected/d8b45469-5973-49bc-b8c2-8ec0dce0e242-kube-api-access-q87h4\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.542887 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.542869 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hkth\" (UniqueName: \"kubernetes.io/projected/24316779-cb78-42df-954b-c36d2b0e097d-kube-api-access-8hkth\") pod \"network-check-source-6f78d59f99-mms8d\" (UID: \"24316779-cb78-42df-954b-c36d2b0e097d\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:11.543001 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.542987 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpmqc\" (UniqueName: \"kubernetes.io/projected/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20-kube-api-access-zpmqc\") pod \"kas-connection-checker-685895c7c9-xtkwb\" (UID: \"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20\") " pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:11.543312 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.543290 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:11.543312 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.543306 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb729\" (UniqueName: \"kubernetes.io/projected/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-kube-api-access-vb729\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:11.543455 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.543392 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.543548 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.543530 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:11.543737 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.543717 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bzv6\" (UniqueName: \"kubernetes.io/projected/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-kube-api-access-6bzv6\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:11.544666 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.544647 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw8wf\" (UniqueName: \"kubernetes.io/projected/4ec29132-5b42-4a06-be85-de3789f00145-kube-api-access-lw8wf\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.569473 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.569453 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:11.580568 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.580551 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:11.589278 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.589260 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:11.602799 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.602638 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:11.612110 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.612083 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:11.641745 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.641526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:11.644649 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.644455 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:11.657595 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.656394 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:11.679220 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.678932 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:11.690373 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.690344 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:11.896861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.896820 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-6z7h7"] Oct 29 22:33:11.898898 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.898236 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-jlpb2"] Oct 29 22:33:11.899648 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.899625 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d"] Oct 29 22:33:11.899930 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:11.899898 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230e7bac_638f_4478_989a_abad8f7213bd.slice/crio-18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827 WatchSource:0}: Error finding container 18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827: Status 404 returned error can't find the container with id 18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827 Oct 29 22:33:11.900585 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:11.900564 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983a2044_473f_48be_8e73_5da2ed82aaf4.slice/crio-deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8 WatchSource:0}: Error finding container deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8: Status 404 returned error can't find the container with id deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8 Oct 29 22:33:11.901280 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:11.901257 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24316779_cb78_42df_954b_c36d2b0e097d.slice/crio-b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa WatchSource:0}: Error finding container b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa: Status 404 returned error can't find the container with id b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa Oct 29 22:33:11.922047 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.922026 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xtkwb"] Oct 29 22:33:11.925913 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:11.925885 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9b3ef6_bbba_450a_a8d5_b4ad21a2fe20.slice/crio-3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e WatchSource:0}: Error finding container 3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e: Status 404 returned error can't find the container with id 3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e Oct 29 22:33:11.946271 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:11.946233 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-m855q"] Oct 29 22:33:12.000704 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:12.000669 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec29132_5b42_4a06_be85_de3789f00145.slice/crio-5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7 WatchSource:0}: Error finding container 5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7: Status 404 returned error can't find the container with id 5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7 Oct 29 22:33:12.002518 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.002497 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p"] Oct 29 22:33:12.004030 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.004009 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d"] Oct 29 22:33:12.015699 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:12.015408 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b45469_5973_49bc_b8c2_8ec0dce0e242.slice/crio-da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109 WatchSource:0}: Error finding container da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109: Status 404 returned error can't find the container with id da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109 Oct 29 22:33:12.016967 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.016873 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:12.016967 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:12.016940 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3a898c_87c6_409d_a70f_f01e8e15d787.slice/crio-f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3 WatchSource:0}: Error finding container f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3: Status 404 returned error can't find the container with id f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3 Oct 29 22:33:12.017342 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.017323 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:12.017787 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.017771 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:12.017993 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.017973 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-8p2vc"] Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021447 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021490 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021552 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021594 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021620 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.021671 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:12.021946 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.021813 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022077 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022120 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.022106772 +0000 UTC m=+31.497451981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022164 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022185 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls podName:5332e2aa-b433-47d6-b3c3-d42e66376eca nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.022178264 +0000 UTC m=+31.497523472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-gwr5t" (UID: "5332e2aa-b433-47d6-b3c3-d42e66376eca") : secret "samples-operator-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022225 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022232 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bf65fd8-phdn2: secret "image-registry-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022251 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls podName:eea9c729-9323-4535-bb3b-f5e67bb28b29 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.022245534 +0000 UTC m=+31.497590742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls") pod "image-registry-66bf65fd8-phdn2" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29") : secret "image-registry-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022298 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022306 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fdff46b97-drfgt: secret "image-registry-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022322 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls podName:05dc76a9-2747-4bd0-907d-a9b2caa3b25d nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.022316988 +0000 UTC m=+31.497662195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls") pod "image-registry-7fdff46b97-drfgt" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d") : secret "image-registry-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022365 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.022359761 +0000 UTC m=+31.497704969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : configmap references non-existent config key: service-ca.crt Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022401 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022418 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.02241266 +0000 UTC m=+31.497757868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022451 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:12.022578 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.022472 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls podName:2ce7d6cc-47fa-4a6a-ba52-5a23efd31019 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.022465208 +0000 UTC m=+31.497810415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lmjqp" (UID: "2ce7d6cc-47fa-4a6a-ba52-5a23efd31019") : secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:12.023307 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:12.023287 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert podName:bba55700-7326-482d-919f-02f3eff316a1 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:13.023269086 +0000 UTC m=+31.498614309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-w564k" (UID: "bba55700-7326-482d-919f-02f3eff316a1") : secret "networking-console-plugin-cert" not found Oct 29 22:33:12.033159 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.033138 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrr8p"] Oct 29 22:33:12.034648 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.034168 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 29 22:33:12.034648 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.034228 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p7zhx\"" Oct 29 22:33:12.034648 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.034271 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-4f7f9\"" Oct 29 22:33:12.034648 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.034371 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-7wdgt\"" Oct 29 22:33:12.034648 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.034438 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 29 22:33:12.045812 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.045767 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk"] Oct 29 22:33:12.052871 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:12.052828 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b59e01_ac2f_4f6d_bc4c_fcbc59afe2a9.slice/crio-c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246 WatchSource:0}: Error finding container c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246: Status 404 returned error can't find the container with id c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246 Oct 29 22:33:12.322189 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.321930 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" event={"ID":"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb","Type":"ContainerStarted","Data":"243f4843e6aedded216f69985397aad912f0de6e9364f4f1269c8cfbf0beea36"} Oct 29 22:33:12.323042 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.323022 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" event={"ID":"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9","Type":"ContainerStarted","Data":"c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246"} Oct 29 22:33:12.324044 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.324023 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" event={"ID":"4ec29132-5b42-4a06-be85-de3789f00145","Type":"ContainerStarted","Data":"5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7"} Oct 29 22:33:12.325065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.325048 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" event={"ID":"ed3a898c-87c6-409d-a70f-f01e8e15d787","Type":"ContainerStarted","Data":"f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3"} Oct 29 22:33:12.326133 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.326114 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" event={"ID":"24316779-cb78-42df-954b-c36d2b0e097d","Type":"ContainerStarted","Data":"b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa"} Oct 29 22:33:12.327432 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.327414 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" event={"ID":"230e7bac-638f-4478-989a-abad8f7213bd","Type":"ContainerStarted","Data":"640fb1549146f5192a038d3087d3266d0674c2919e0627e4c84285364ea938d5"} Oct 29 22:33:12.327508 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.327445 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" event={"ID":"230e7bac-638f-4478-989a-abad8f7213bd","Type":"ContainerStarted","Data":"18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827"} Oct 29 22:33:12.328830 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.328812 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" event={"ID":"983a2044-473f-48be-8e73-5da2ed82aaf4","Type":"ContainerStarted","Data":"e4e61c631ab6f1a6465da3db17f11f9699ef7b16d4f72428d63346bb4e974a78"} Oct 29 22:33:12.328919 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.328863 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" event={"ID":"983a2044-473f-48be-8e73-5da2ed82aaf4","Type":"ContainerStarted","Data":"deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8"} Oct 29 22:33:12.329964 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.329943 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrr8p" event={"ID":"8db65a63-2853-43db-8877-83a0d26c2de2","Type":"ContainerStarted","Data":"43ab2caceddd95f7c63da7115cfb15ff9df0fb3216ffb913aa195b53728d2888"} Oct 29 22:33:12.331016 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.330994 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" event={"ID":"d8b45469-5973-49bc-b8c2-8ec0dce0e242","Type":"ContainerStarted","Data":"da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109"} Oct 29 22:33:12.332192 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.332172 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" event={"ID":"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20","Type":"ContainerStarted","Data":"d125c8309f1068d0c0b99c0b768426178c2401ef056bcaf6fb9b6eb5377c779a"} Oct 29 22:33:12.332284 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.332202 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" event={"ID":"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20","Type":"ContainerStarted","Data":"3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e"} Oct 29 22:33:12.366646 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.366602 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" podStartSLOduration=1178.366586022 podStartE2EDuration="19m38.366586022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:13:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:12.365453629 +0000 UTC m=+30.840798861" watchObservedRunningTime="2026-10-29 22:33:12.366586022 +0000 UTC m=+30.841931256" Oct 29 22:33:12.405993 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.405939 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" podStartSLOduration=1178.405922279 podStartE2EDuration="19m38.405922279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:13:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:12.404434263 +0000 UTC m=+30.879779496" watchObservedRunningTime="2026-10-29 22:33:12.405922279 +0000 UTC m=+30.881267510" Oct 29 22:33:12.445024 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.444979 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" podStartSLOduration=1178.444962451 podStartE2EDuration="19m38.444962451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:13:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:12.444211862 +0000 UTC m=+30.919557092" watchObservedRunningTime="2026-10-29 22:33:12.444962451 +0000 UTC m=+30.920307683" Oct 29 22:33:12.570569 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.570525 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:12.705811 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.705713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:12.918231 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:12.918198 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030160 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030204 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030237 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030299 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030323 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.030443 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.030595 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.030656 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.030638532 +0000 UTC m=+33.505983741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031065 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031111 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031096316 +0000 UTC m=+33.506441530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031185 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031175079 +0000 UTC m=+33.506520290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : configmap references non-existent config key: service-ca.crt Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031247 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:13.039172 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031277 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls podName:2ce7d6cc-47fa-4a6a-ba52-5a23efd31019 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031268525 +0000 UTC m=+33.506613741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lmjqp" (UID: "2ce7d6cc-47fa-4a6a-ba52-5a23efd31019") : secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031332 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031360 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert podName:bba55700-7326-482d-919f-02f3eff316a1 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031350787 +0000 UTC m=+33.506696002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-w564k" (UID: "bba55700-7326-482d-919f-02f3eff316a1") : secret "networking-console-plugin-cert" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031414 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031446 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls podName:5332e2aa-b433-47d6-b3c3-d42e66376eca nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031436486 +0000 UTC m=+33.506781695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-gwr5t" (UID: "5332e2aa-b433-47d6-b3c3-d42e66376eca") : secret "samples-operator-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031506 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031516 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bf65fd8-phdn2: secret "image-registry-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031554 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls podName:eea9c729-9323-4535-bb3b-f5e67bb28b29 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031544163 +0000 UTC m=+33.506889386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls") pod "image-registry-66bf65fd8-phdn2" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29") : secret "image-registry-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031609 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031617 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fdff46b97-drfgt: secret "image-registry-tls" not found Oct 29 22:33:13.040299 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:13.031649 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls podName:05dc76a9-2747-4bd0-907d-a9b2caa3b25d nodeName:}" failed. No retries permitted until 2026-10-29 22:33:15.031637616 +0000 UTC m=+33.506982827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls") pod "image-registry-7fdff46b97-drfgt" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d") : secret "image-registry-tls" not found Oct 29 22:33:13.228489 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.228415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:13.311415 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:13.311315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052521 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052636 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.052720 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.052695714 +0000 UTC m=+37.528040925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : configmap references non-existent config key: service-ca.crt Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052793 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.052800 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.052885 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert podName:bba55700-7326-482d-919f-02f3eff316a1 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.052865652 +0000 UTC m=+37.528210879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-w564k" (UID: "bba55700-7326-482d-919f-02f3eff316a1") : secret "networking-console-plugin-cert" not found Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.052949 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.052962 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bf65fd8-phdn2: secret "image-registry-tls" not found Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.052975 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053005 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls podName:eea9c729-9323-4535-bb3b-f5e67bb28b29 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.05299073 +0000 UTC m=+37.528335943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls") pod "image-registry-66bf65fd8-phdn2" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29") : secret "image-registry-tls" not found Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.053049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053053 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:15.053492 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053115 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053134 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.053122366 +0000 UTC m=+37.528467577 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053156 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls podName:2ce7d6cc-47fa-4a6a-ba52-5a23efd31019 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.053141805 +0000 UTC m=+37.528487021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lmjqp" (UID: "2ce7d6cc-47fa-4a6a-ba52-5a23efd31019") : secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053217 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053229 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fdff46b97-drfgt: secret "image-registry-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053273 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls podName:05dc76a9-2747-4bd0-907d-a9b2caa3b25d nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.053261897 +0000 UTC m=+37.528607111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls") pod "image-registry-7fdff46b97-drfgt" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d") : secret "image-registry-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053329 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053359 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.053349388 +0000 UTC m=+37.528694629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053420 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 22:33:15.054810 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.053456 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls podName:5332e2aa-b433-47d6-b3c3-d42e66376eca nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.053445713 +0000 UTC m=+37.528790930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-gwr5t" (UID: "5332e2aa-b433-47d6-b3c3-d42e66376eca") : secret "samples-operator-tls" not found Oct 29 22:33:15.349224 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.349141 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" event={"ID":"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9","Type":"ContainerStarted","Data":"3aee0211900ead934767ac39bd7bd7fa71aac8a25238e49fbb52b41bea820456"} Oct 29 22:33:15.391428 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.390925 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" podStartSLOduration=1144.940436122 podStartE2EDuration="19m7.39090957s" totalImagesPullingTime="2.450473448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:15.389928472 +0000 UTC m=+33.865273702" watchObservedRunningTime="2026-10-29 22:33:15.39090957 +0000 UTC m=+33.866254801" Oct 29 22:33:15.489195 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.489111 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:15.589982 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.589947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:15.760460 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.760426 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:15.765696 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.765664 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:15.862106 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.860951 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:15.862106 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.861171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:15.862106 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.861325 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 22:33:15.862106 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:15.861390 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:47.861372764 +0000 UTC m=+66.336717995 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : secret "metrics-daemon-secret" not found Oct 29 22:33:15.866113 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.866090 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:15.968862 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.968651 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:15.998927 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:15.988755 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:16.290878 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.290785 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-65b9p"] Oct 29 22:33:16.294243 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:16.294211 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f37d2fc_3cf7_4105_9c73_bf532a71af91.slice/crio-84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b WatchSource:0}: Error finding container 84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b: Status 404 returned error can't find the container with id 84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b Oct 29 22:33:16.338870 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.338827 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rh76g"] Oct 29 22:33:16.342558 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:16.342529 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa67ab99_04d4_4f83_a61b_d6f8fe0b0032.slice/crio-7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f WatchSource:0}: Error finding container 7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f: Status 404 returned error can't find the container with id 7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f Oct 29 22:33:16.354449 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.354419 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-65b9p" event={"ID":"3f37d2fc-3cf7-4105-9c73-bf532a71af91","Type":"ContainerStarted","Data":"84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b"} Oct 29 22:33:16.356685 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.356663 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/0.log" Oct 29 22:33:16.356769 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.356697 2574 generic.go:356] "Generic (PLEG): container finished" podID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" containerID="2f68d61aa56d2724972485bf94fa14e31b077fb134567d124bcac55e77c1aa73" exitCode=255 Oct 29 22:33:16.356769 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.356759 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" event={"ID":"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb","Type":"ContainerDied","Data":"2f68d61aa56d2724972485bf94fa14e31b077fb134567d124bcac55e77c1aa73"} Oct 29 22:33:16.357103 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.357082 2574 scope.go:122] "RemoveContainer" containerID="2f68d61aa56d2724972485bf94fa14e31b077fb134567d124bcac55e77c1aa73" Oct 29 22:33:16.358804 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:16.358756 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rh76g" event={"ID":"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032","Type":"ContainerStarted","Data":"7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f"} Oct 29 22:33:17.367923 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.367269 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" event={"ID":"ed3a898c-87c6-409d-a70f-f01e8e15d787","Type":"ContainerStarted","Data":"f9d3e2835828fd3d57f02bdc7028af74cff9a6e1b251467c3d0e3f75c0f364d7"} Oct 29 22:33:17.369980 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.369954 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" event={"ID":"d8b45469-5973-49bc-b8c2-8ec0dce0e242","Type":"ContainerStarted","Data":"814f49e837d5fc2a5c057f4343d4d6c092c26b5c13039857219074b4ca657c59"} Oct 29 22:33:17.374348 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.374329 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/1.log" Oct 29 22:33:17.374798 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.374778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/0.log" Oct 29 22:33:17.374890 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.374814 2574 generic.go:356] "Generic (PLEG): container finished" podID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" exitCode=255 Oct 29 22:33:17.374950 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.374907 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" event={"ID":"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb","Type":"ContainerDied","Data":"42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c"} Oct 29 22:33:17.374950 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.374947 2574 scope.go:122] "RemoveContainer" containerID="2f68d61aa56d2724972485bf94fa14e31b077fb134567d124bcac55e77c1aa73" Oct 29 22:33:17.375340 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.375325 2574 scope.go:122] "RemoveContainer" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" Oct 29 22:33:17.375532 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:17.375514 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-m855q_openshift-console-operator(5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" Oct 29 22:33:17.384649 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.383864 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" event={"ID":"4ec29132-5b42-4a06-be85-de3789f00145","Type":"ContainerStarted","Data":"475c87c0fad0bba0eed642494df7438b7bbce2c4c92289dd8927b2b953089096"} Oct 29 22:33:17.459866 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.458906 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" podStartSLOduration=1144.353168345 podStartE2EDuration="19m9.458823827s" totalImagesPullingTime="5.105655482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:17.458481663 +0000 UTC m=+35.933826893" watchObservedRunningTime="2026-10-29 22:33:17.458823827 +0000 UTC m=+35.934169060" Oct 29 22:33:17.460662 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.460496 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" podStartSLOduration=1151.368772713 podStartE2EDuration="19m16.460482226s" totalImagesPullingTime="5.091709513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:17.414923067 +0000 UTC m=+35.890268299" watchObservedRunningTime="2026-10-29 22:33:17.460482226 +0000 UTC m=+35.935827462" Oct 29 22:33:17.541330 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:17.541238 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" podStartSLOduration=1145.450040233 podStartE2EDuration="19m10.541215282s" totalImagesPullingTime="5.091175049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:14:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:17.540513112 +0000 UTC m=+36.015858342" watchObservedRunningTime="2026-10-29 22:33:17.541215282 +0000 UTC m=+36.016560516" Oct 29 22:33:18.104179 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.104095 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:18.141018 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.140987 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh"] Oct 29 22:33:18.171459 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.170958 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb"] Oct 29 22:33:18.173255 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.172877 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:18.186373 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.186349 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh"] Oct 29 22:33:18.186486 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.186394 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb"] Oct 29 22:33:18.186486 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.186468 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:18.191874 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.191016 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:18.191874 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.191640 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-shpd4\"" Oct 29 22:33:18.192574 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.192542 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 29 22:33:18.193881 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.193832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbrmd\" (UniqueName: \"kubernetes.io/projected/901d2e7b-558b-4fc4-833b-a71699d02494-kube-api-access-bbrmd\") pod \"migrator-66df5d496b-n6zwh\" (UID: \"901d2e7b-558b-4fc4-833b-a71699d02494\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:18.194033 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.194012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvcvp\" (UniqueName: \"kubernetes.io/projected/8978b075-27a6-4c34-b4d4-36270f768a01-kube-api-access-nvcvp\") pod \"migrator-66df5d496b-kjspb\" (UID: \"8978b075-27a6-4c34-b4d4-36270f768a01\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:18.229401 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.229371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:18.246300 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.246274 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tdvz7"] Oct 29 22:33:18.265271 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.265213 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.283668 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.283604 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tdvz7"] Oct 29 22:33:18.289872 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.287911 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-s72v6-5hwwc"] Oct 29 22:33:18.293876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.290654 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 29 22:33:18.293876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.291436 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 29 22:33:18.293876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.291583 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8pxqn\"" Oct 29 22:33:18.294861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.294552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsb72\" (UniqueName: \"kubernetes.io/projected/3374d1d7-08e1-409a-8361-cf995598345b-kube-api-access-zsb72\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.294861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.294594 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.294861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.294683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvcvp\" (UniqueName: \"kubernetes.io/projected/8978b075-27a6-4c34-b4d4-36270f768a01-kube-api-access-nvcvp\") pod \"migrator-66df5d496b-kjspb\" (UID: \"8978b075-27a6-4c34-b4d4-36270f768a01\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:18.294861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.294753 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.294861 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.294795 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3374d1d7-08e1-409a-8361-cf995598345b-crio-socket\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.296119 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.294832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3374d1d7-08e1-409a-8361-cf995598345b-data-volume\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.296119 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.295940 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbrmd\" (UniqueName: \"kubernetes.io/projected/901d2e7b-558b-4fc4-833b-a71699d02494-kube-api-access-bbrmd\") pod \"migrator-66df5d496b-n6zwh\" (UID: \"901d2e7b-558b-4fc4-833b-a71699d02494\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:18.305152 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.305129 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-s72v6-5hwwc"] Oct 29 22:33:18.305260 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.305250 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.317185 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.317168 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvcvp\" (UniqueName: \"kubernetes.io/projected/8978b075-27a6-4c34-b4d4-36270f768a01-kube-api-access-nvcvp\") pod \"migrator-66df5d496b-kjspb\" (UID: \"8978b075-27a6-4c34-b4d4-36270f768a01\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:18.317656 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.317636 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbrmd\" (UniqueName: \"kubernetes.io/projected/901d2e7b-558b-4fc4-833b-a71699d02494-kube-api-access-bbrmd\") pod \"migrator-66df5d496b-n6zwh\" (UID: \"901d2e7b-558b-4fc4-833b-a71699d02494\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:18.327819 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.327793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:18.389863 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.389779 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrr8p" event={"ID":"8db65a63-2853-43db-8877-83a0d26c2de2","Type":"ContainerStarted","Data":"4eadf432f5a9cc2d5272e41057fe0644478e8ec473740a5a483c91c2f050d406"} Oct 29 22:33:18.391072 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.391054 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rhrpc\"" Oct 29 22:33:18.392093 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.392063 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/1.log" Oct 29 22:33:18.392339 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.392326 2574 scope.go:122] "RemoveContainer" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" Oct 29 22:33:18.392502 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.392485 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-m855q_openshift-console-operator(5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" Oct 29 22:33:18.395420 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.395399 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" event={"ID":"24316779-cb78-42df-954b-c36d2b0e097d","Type":"ContainerStarted","Data":"d02bb0341513e15b95171cca6a5f62bb029d8465253959700559aa85ca1d12e3"} Oct 29 22:33:18.396330 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.396302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3374d1d7-08e1-409a-8361-cf995598345b-data-volume\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.396674 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.396652 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3374d1d7-08e1-409a-8361-cf995598345b-data-volume\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.397012 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.396761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsb72\" (UniqueName: \"kubernetes.io/projected/3374d1d7-08e1-409a-8361-cf995598345b-kube-api-access-zsb72\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.397012 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.396792 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.397012 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.396827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.397012 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.396945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.397266 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.397044 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:18.397266 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.397059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.397266 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.397120 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 22:33:18.397266 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.397168 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:18.897151605 +0000 UTC m=+37.372496831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : configmap "kube-rbac-proxy" not found Oct 29 22:33:18.397266 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.397205 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:18.897195028 +0000 UTC m=+37.372540236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:18.397266 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.397241 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.397543 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.397297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.397543 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.397475 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3374d1d7-08e1-409a-8361-cf995598345b-crio-socket\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.397633 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.397552 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3374d1d7-08e1-409a-8361-cf995598345b-crio-socket\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.408830 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.408772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:18.418733 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.418711 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsb72\" (UniqueName: \"kubernetes.io/projected/3374d1d7-08e1-409a-8361-cf995598345b-kube-api-access-zsb72\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.430388 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.430345 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rrr8p" podStartSLOduration=1.7536955170000001 podStartE2EDuration="7.430330391s" totalImagesPullingTime="5.676634874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:18.429982119 +0000 UTC m=+36.905327350" watchObservedRunningTime="2026-10-29 22:33:18.430330391 +0000 UTC m=+36.905675625" Oct 29 22:33:18.498804 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.498772 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.498978 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.498888 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.499031 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.499005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.499083 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.499063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.500208 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.500183 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.501515 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.501200 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.501515 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.501247 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.507956 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.507906 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:18.511490 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.511448 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" podStartSLOduration=1143.684721641 podStartE2EDuration="19m9.511437486s" totalImagesPullingTime="5.826715845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:18.511255928 +0000 UTC m=+36.986601160" watchObservedRunningTime="2026-10-29 22:33:18.511437486 +0000 UTC m=+36.986782715" Oct 29 22:33:18.521814 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.521759 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:18.524331 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.524311 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.645067 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.644823 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:18.771949 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.771907 2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-6564h"] Oct 29 22:33:18.798400 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.798368 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-6564h"] Oct 29 22:33:18.798566 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.798468 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.805660 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.805626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-key\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.805816 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.805670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-cabundle\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.805816 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.805780 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxs2w\" (UniqueName: \"kubernetes.io/projected/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-kube-api-access-cxs2w\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.814876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.814636 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 29 22:33:18.814876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.814661 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 29 22:33:18.814876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.814682 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 29 22:33:18.814876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.814636 2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lsqkb\"" Oct 29 22:33:18.814876 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.814695 2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 29 22:33:18.852935 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.852912 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh"] Oct 29 22:33:18.855755 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:18.855729 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901d2e7b_558b_4fc4_833b_a71699d02494.slice/crio-82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5 WatchSource:0}: Error finding container 82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5: Status 404 returned error can't find the container with id 82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5 Oct 29 22:33:18.892242 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.892219 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb"] Oct 29 22:33:18.894416 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:18.894362 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8978b075_27a6_4c34_b4d4_36270f768a01.slice/crio-da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465 WatchSource:0}: Error finding container da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465: Status 404 returned error can't find the container with id da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465 Oct 29 22:33:18.907283 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.907255 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-key\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.907383 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.907292 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-cabundle\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.907383 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.907331 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.907486 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.907436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:18.907486 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.907465 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxs2w\" (UniqueName: \"kubernetes.io/projected/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-kube-api-access-cxs2w\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.907580 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.907536 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:18.907630 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.907594 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.907575929 +0000 UTC m=+38.382921151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:18.907686 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.907641 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 22:33:18.907738 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:18.907701 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:19.907683439 +0000 UTC m=+38.383028660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : configmap "kube-rbac-proxy" not found Oct 29 22:33:18.908258 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.908240 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-cabundle\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.910231 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.910211 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-key\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.928168 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.928144 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxs2w\" (UniqueName: \"kubernetes.io/projected/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-kube-api-access-cxs2w\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:18.979065 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:18.979043 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-s72v6-5hwwc"] Oct 29 22:33:18.981412 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:18.981386 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56290f6c_9bcf_44ca_b44c_aa29ac9cbcc7.slice/crio-e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9 WatchSource:0}: Error finding container e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9: Status 404 returned error can't find the container with id e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9 Oct 29 22:33:19.109859 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.109812 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:19.109859 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.109870 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:19.110096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.109927 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:19.110096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.109957 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:19.110096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.109994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:19.110096 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.110015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:19.110216 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110148 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:19.110216 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110158 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:19.110216 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110170 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fdff46b97-drfgt: secret "image-registry-tls" not found Oct 29 22:33:19.110216 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110195 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110172741 +0000 UTC m=+45.585517967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : configmap references non-existent config key: service-ca.crt Oct 29 22:33:19.110346 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110225 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110213035 +0000 UTC m=+45.585558249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:19.110346 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110234 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:19.110346 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110238 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:19.110346 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110240 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls podName:05dc76a9-2747-4bd0-907d-a9b2caa3b25d nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110232468 +0000 UTC m=+45.585577682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls") pod "image-registry-7fdff46b97-drfgt" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d") : secret "image-registry-tls" not found Oct 29 22:33:19.110346 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110278 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110262927 +0000 UTC m=+45.585608140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:19.110346 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110293 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls podName:2ce7d6cc-47fa-4a6a-ba52-5a23efd31019 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110283938 +0000 UTC m=+45.585629149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lmjqp" (UID: "2ce7d6cc-47fa-4a6a-ba52-5a23efd31019") : secret "cluster-monitoring-operator-tls" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110351 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.110374 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110384 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert podName:bba55700-7326-482d-919f-02f3eff316a1 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.11037384 +0000 UTC m=+45.585719067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-w564k" (UID: "bba55700-7326-482d-919f-02f3eff316a1") : secret "networking-console-plugin-cert" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.110410 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110430 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110470 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls podName:5332e2aa-b433-47d6-b3c3-d42e66376eca nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110460493 +0000 UTC m=+45.585805706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-gwr5t" (UID: "5332e2aa-b433-47d6-b3c3-d42e66376eca") : secret "samples-operator-tls" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110505 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110514 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bf65fd8-phdn2: secret "image-registry-tls" not found Oct 29 22:33:19.110529 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.110533 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls podName:eea9c729-9323-4535-bb3b-f5e67bb28b29 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.110527304 +0000 UTC m=+45.585872541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls") pod "image-registry-66bf65fd8-phdn2" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29") : secret "image-registry-tls" not found Oct 29 22:33:19.137800 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.137765 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:19.238694 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.238646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:19.399795 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.399758 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" event={"ID":"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7","Type":"ContainerStarted","Data":"9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc"} Oct 29 22:33:19.400248 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.399815 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" event={"ID":"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7","Type":"ContainerStarted","Data":"e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9"} Oct 29 22:33:19.401520 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.401495 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" event={"ID":"8978b075-27a6-4c34-b4d4-36270f768a01","Type":"ContainerStarted","Data":"da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465"} Oct 29 22:33:19.402901 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.402874 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" event={"ID":"901d2e7b-558b-4fc4-833b-a71699d02494","Type":"ContainerStarted","Data":"82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5"} Oct 29 22:33:19.442095 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.441915 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" podStartSLOduration=1.441901597 podStartE2EDuration="1.441901597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:19.440472342 +0000 UTC m=+37.915817573" watchObservedRunningTime="2026-10-29 22:33:19.441901597 +0000 UTC m=+37.917246827" Oct 29 22:33:19.479364 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.479328 2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-6564h"] Oct 29 22:33:19.483108 ip-10-0-136-228 kubenswrapper[2574]: W1029 22:33:19.483084 2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfffc6a30_b103_428e_a73e_7d7d03a4e2ff.slice/crio-00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51 WatchSource:0}: Error finding container 00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51: Status 404 returned error can't find the container with id 00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51 Oct 29 22:33:19.680072 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.680007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:19.918457 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.918427 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:19.918649 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:19.918529 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:19.918707 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.918692 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 22:33:19.918963 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.918756 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:21.918737103 +0000 UTC m=+40.394082329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : configmap "kube-rbac-proxy" not found Oct 29 22:33:19.918963 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.918893 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:19.918963 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:19.918944 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:21.918930402 +0000 UTC m=+40.394275627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:20.420605 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.419782 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rh76g" event={"ID":"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032","Type":"ContainerStarted","Data":"58ea4ecaa583bb00343fd52b66f7beea916c2319f3bfc6e4015bc4547ef61458"} Oct 29 22:33:20.420605 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.420546 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:20.423816 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.423790 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" event={"ID":"fffc6a30-b103-428e-a73e-7d7d03a4e2ff","Type":"ContainerStarted","Data":"0be628b7ed1bd437471273a660d13c1986a2ce20f6bad6b371249c00dc0395af"} Oct 29 22:33:20.423975 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.423828 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" event={"ID":"fffc6a30-b103-428e-a73e-7d7d03a4e2ff","Type":"ContainerStarted","Data":"00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51"} Oct 29 22:33:20.461869 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.461323 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:20.521587 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.521514 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rh76g" podStartSLOduration=35.314662807 podStartE2EDuration="38.521495334s" totalImagesPullingTime="3.206832527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:20.467306315 +0000 UTC m=+38.942651548" watchObservedRunningTime="2026-10-29 22:33:20.521495334 +0000 UTC m=+38.996840565" Oct 29 22:33:20.521866 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.521804 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" podStartSLOduration=2.521792852 podStartE2EDuration="2.521792852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:20.508880464 +0000 UTC m=+38.984225691" watchObservedRunningTime="2026-10-29 22:33:20.521792852 +0000 UTC m=+38.997138083" Oct 29 22:33:20.621633 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.621598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:20.723291 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.723240 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:20.822532 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.822501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:20.975139 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:20.975086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:21.243260 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.243183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9655m_1ac5c5e2-a666-4cbf-9a73-677929dc640e/dns-node-resolver/0.log" Oct 29 22:33:21.428342 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.428304 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" event={"ID":"8978b075-27a6-4c34-b4d4-36270f768a01","Type":"ContainerStarted","Data":"0432649153e1f839830d7bb01d8e314cef1e5db2de5f17b97d22cdc0197e329b"} Oct 29 22:33:21.430145 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.430119 2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" event={"ID":"901d2e7b-558b-4fc4-833b-a71699d02494","Type":"ContainerStarted","Data":"be9d039ffb8d624627c1524d48ccaa53c1ad15121a2687089975e8ca51ab2b63"} Oct 29 22:33:21.468439 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.468395 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" podStartSLOduration=1.624959864 podStartE2EDuration="3.468381173s" totalImagesPullingTime="1.843421309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:21.467487034 +0000 UTC m=+39.942832264" watchObservedRunningTime="2026-10-29 22:33:21.468381173 +0000 UTC m=+39.943726419" Oct 29 22:33:21.508271 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.508166 2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" podStartSLOduration=1.4194891840000001 podStartE2EDuration="3.508148531s" totalImagesPullingTime="2.088659347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:21.505969672 +0000 UTC m=+39.981314903" watchObservedRunningTime="2026-10-29 22:33:21.508148531 +0000 UTC m=+39.983493764" Oct 29 22:33:21.612900 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.612863 2574 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:21.613067 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.612925 2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:21.613300 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.613285 2574 scope.go:122] "RemoveContainer" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" Oct 29 22:33:21.613493 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:21.613472 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-m855q_openshift-console-operator(5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" Oct 29 22:33:21.848492 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.848422 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkr9j_c7881bd2-9f24-4c58-9a77-68cc091ee9ab/node-ca/0.log" Oct 29 22:33:21.944039 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.943557 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:21.944039 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:21.943669 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:21.944039 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:21.943744 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:21.944039 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:21.943802 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:25.94378468 +0000 UTC m=+44.419129891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:21.944039 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:21.943808 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 22:33:21.944039 ip-10-0-136-228 kubenswrapper[2574]: E1029 22:33:21.943882 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:25.943869034 +0000 UTC m=+44.419214255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : configmap "kube-rbac-proxy" not found Oct 29 22:33:22.014857 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.014785 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:22.242752 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.242721 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrr8p_8db65a63-2853-43db-8877-83a0d26c2de2/serve-healthcheck-canary/0.log" Oct 29 22:33:22.357109 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.357080 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:22.443964 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.443934 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-kjspb_8978b075-27a6-4c34-b4d4-36270f768a01/migrator/0.log" Oct 29 22:33:22.458935 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.458906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:22.563674 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.563647 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:22.645704 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.645683 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-n6zwh_901d2e7b-558b-4fc4-833b-a71699d02494/migrator/0.log" Oct 29 22:33:22.768331 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.768304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:22.833132 ip-10-0-136-228 kubenswrapper[2574]: I1029 22:33:22.832981 2574 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 22:33:22.833157 ip-10-0-136-228 systemd[1]: Stopping Kubernetes Kubelet... Oct 29 22:33:22.844272 ip-10-0-136-228 systemd[1]: kubelet.service: Deactivated successfully. Oct 29 22:33:22.844436 ip-10-0-136-228 systemd[1]: Stopped Kubernetes Kubelet. Oct 29 22:33:22.844601 ip-10-0-136-228 systemd[1]: kubelet.service: Consumed 3.131s CPU time. Oct 29 22:33:22.856275 ip-10-0-136-228 systemd[1]: Starting Kubernetes Kubelet... Oct 29 22:33:22.947713 ip-10-0-136-228 kubenswrapper[5030]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 22:33:22.947713 ip-10-0-136-228 kubenswrapper[5030]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 29 22:33:22.947713 ip-10-0-136-228 kubenswrapper[5030]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 22:33:22.947713 ip-10-0-136-228 kubenswrapper[5030]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 22:33:22.949720 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949645 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.949720 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949721 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949725 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949729 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949731 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949734 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949736 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949739 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949742 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949744 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949747 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949749 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949752 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949754 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949757 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949760 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949763 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949765 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949768 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949771 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.949794 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949774 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949776 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949780 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949782 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949785 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949787 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949790 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949792 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949802 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949806 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949809 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949812 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949814 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949817 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949820 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949822 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949825 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949827 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949830 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949832 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.950295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949849 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949853 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949857 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949861 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949866 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949870 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949874 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949878 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949881 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949884 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949887 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949890 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949892 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949896 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949898 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949901 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949903 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949906 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949909 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.950793 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949912 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949915 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949918 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949920 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949923 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949925 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949928 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949930 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949933 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949935 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949938 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949940 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949943 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949945 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949948 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949950 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949953 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949955 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949958 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949960 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.951291 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949963 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949966 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949968 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949971 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949974 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949977 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949981 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949986 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949989 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949993 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949995 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.949998 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950001 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950004 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950006 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950010 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950013 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950015 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950018 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.951882 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950020 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950023 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950025 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950028 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950030 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950033 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950035 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950038 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950796 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950803 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950807 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950810 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950813 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950816 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950819 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950821 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950824 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950827 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950830 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.952433 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950833 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950853 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950857 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950861 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950865 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950869 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950872 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950875 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950877 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950880 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950884 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950887 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950889 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950892 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950894 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950897 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950899 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950902 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950904 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950907 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.952953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950909 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950912 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950915 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950917 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950920 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950922 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950925 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950928 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950930 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950933 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950936 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950938 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950941 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950943 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950946 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950948 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950951 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950953 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950956 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950958 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.953610 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950961 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950963 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950966 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950969 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950971 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950974 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950976 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950979 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950981 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950983 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950986 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950988 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950990 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950994 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.950999 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951002 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951005 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951007 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951010 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.954427 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951015 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951018 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951020 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951024 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951027 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951030 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951033 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951035 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951038 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951040 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951043 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951046 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951048 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951051 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951053 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951056 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951059 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951062 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951065 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.954956 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951067 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951070 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951073 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951075 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951078 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951080 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951083 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951085 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951088 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951090 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951093 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951095 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951098 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951102 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951104 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951106 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951109 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951593 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951598 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951601 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.955437 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951604 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951606 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951609 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951612 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951615 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951617 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951620 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951622 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951625 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951628 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951632 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951635 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951637 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951640 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951642 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951645 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951647 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951650 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951652 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951655 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.955996 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951657 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951660 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951662 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951665 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951667 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951671 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951674 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951676 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951679 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951681 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951684 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951686 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951689 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951691 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951696 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951700 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951703 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951707 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951710 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.956727 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951713 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951716 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951718 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951721 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951724 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951726 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951729 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951731 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951733 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951736 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951738 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951741 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951743 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951745 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951748 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951750 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951752 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951758 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951763 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.957589 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951765 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951768 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951771 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951773 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951776 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951779 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951781 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951784 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951786 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951789 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951791 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951793 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951796 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951799 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951801 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951803 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951806 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951808 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951811 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.958415 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951814 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951816 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951819 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951821 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951824 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951826 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951829 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951831 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951847 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951852 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951856 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951861 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951866 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951870 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951874 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951877 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951880 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951884 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951888 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951891 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.959275 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951894 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951896 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951899 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951902 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951904 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.951907 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952019 5030 flags.go:64] FLAG: --address="0.0.0.0" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952028 5030 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952036 5030 flags.go:64] FLAG: --anonymous-auth="true" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952043 5030 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952050 5030 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952054 5030 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952062 5030 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952066 5030 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952069 5030 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952072 5030 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952075 5030 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952078 5030 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952081 5030 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952084 5030 flags.go:64] FLAG: --cgroup-root="" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952087 5030 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952090 5030 flags.go:64] FLAG: --client-ca-file="" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952093 5030 flags.go:64] FLAG: --cloud-provider="external" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952098 5030 flags.go:64] FLAG: --cluster-dns="[]" Oct 29 22:33:22.960183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952103 5030 flags.go:64] FLAG: --cluster-domain="" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952106 5030 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952109 5030 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952112 5030 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952115 5030 flags.go:64] FLAG: --container-log-max-files="5" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952119 5030 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952122 5030 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952125 5030 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952128 5030 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952131 5030 flags.go:64] FLAG: --contention-profiling="false" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952134 5030 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952137 5030 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952140 5030 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952143 5030 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952147 5030 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952150 5030 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952153 5030 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952156 5030 flags.go:64] FLAG: --enable-load-reader="false" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952159 5030 flags.go:64] FLAG: --enable-server="true" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952162 5030 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952166 5030 flags.go:64] FLAG: --event-burst="100" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952169 5030 flags.go:64] FLAG: --event-qps="50" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952172 5030 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952175 5030 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 29 22:33:22.961254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952178 5030 flags.go:64] FLAG: --eviction-hard="" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952182 5030 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952185 5030 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952188 5030 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952191 5030 flags.go:64] FLAG: --eviction-soft="" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952195 5030 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952201 5030 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952207 5030 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952212 5030 flags.go:64] FLAG: --experimental-mounter-path="" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952216 5030 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952220 5030 flags.go:64] FLAG: --fail-swap-on="true" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952222 5030 flags.go:64] FLAG: --feature-gates="" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952226 5030 flags.go:64] FLAG: --file-check-frequency="20s" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952230 5030 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952233 5030 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952236 5030 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952239 5030 flags.go:64] FLAG: --healthz-port="10248" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952242 5030 flags.go:64] FLAG: --help="false" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952245 5030 flags.go:64] FLAG: --hostname-override="ip-10-0-136-228.ec2.internal" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952248 5030 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952251 5030 flags.go:64] FLAG: --http-check-frequency="20s" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952253 5030 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952257 5030 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 29 22:33:22.962331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952260 5030 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952263 5030 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952266 5030 flags.go:64] FLAG: --image-service-endpoint="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952269 5030 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952271 5030 flags.go:64] FLAG: --kube-api-burst="100" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952274 5030 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952277 5030 flags.go:64] FLAG: --kube-api-qps="50" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952281 5030 flags.go:64] FLAG: --kube-reserved="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952284 5030 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952287 5030 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952289 5030 flags.go:64] FLAG: --kubelet-cgroups="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952292 5030 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952295 5030 flags.go:64] FLAG: --lock-file="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952298 5030 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952301 5030 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952304 5030 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952310 5030 flags.go:64] FLAG: --log-json-split-stream="false" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952313 5030 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952316 5030 flags.go:64] FLAG: --log-text-split-stream="false" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952319 5030 flags.go:64] FLAG: --logging-format="text" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952322 5030 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952325 5030 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952328 5030 flags.go:64] FLAG: --manifest-url="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952331 5030 flags.go:64] FLAG: --manifest-url-header="" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952335 5030 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 29 22:33:22.963382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952338 5030 flags.go:64] FLAG: --max-open-files="1000000" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952342 5030 flags.go:64] FLAG: --max-pods="110" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952345 5030 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952348 5030 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952350 5030 flags.go:64] FLAG: --memory-manager-policy="None" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952353 5030 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952356 5030 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952359 5030 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952362 5030 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952369 5030 flags.go:64] FLAG: --node-status-max-images="50" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952372 5030 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952374 5030 flags.go:64] FLAG: --oom-score-adj="-999" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952378 5030 flags.go:64] FLAG: --pod-cidr="" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952381 5030 flags.go:64] FLAG: --pod-manifest-path="" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952383 5030 flags.go:64] FLAG: --pod-max-pids="-1" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952387 5030 flags.go:64] FLAG: --pods-per-core="0" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952390 5030 flags.go:64] FLAG: --port="10250" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952393 5030 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952396 5030 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d56abb05ac7d75ce" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952399 5030 flags.go:64] FLAG: --qos-reserved="" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952402 5030 flags.go:64] FLAG: --read-only-port="10255" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952405 5030 flags.go:64] FLAG: --register-node="true" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952407 5030 flags.go:64] FLAG: --register-with-taints="" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952413 5030 flags.go:64] FLAG: --registry-burst="10" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952415 5030 flags.go:64] FLAG: --registry-qps="5" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952419 5030 flags.go:64] FLAG: --reserved-cpus="" Oct 29 22:33:22.964455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952421 5030 flags.go:64] FLAG: --reserved-memory="" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952425 5030 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952428 5030 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952432 5030 flags.go:64] FLAG: --rotate-certificates="false" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952435 5030 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952438 5030 flags.go:64] FLAG: --runonce="false" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952441 5030 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952444 5030 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952447 5030 flags.go:64] FLAG: --seccomp-default="false" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952450 5030 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952453 5030 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952456 5030 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952460 5030 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952463 5030 flags.go:64] FLAG: --storage-driver-password="root" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952465 5030 flags.go:64] FLAG: --storage-driver-secure="false" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952468 5030 flags.go:64] FLAG: --storage-driver-table="stats" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952471 5030 flags.go:64] FLAG: --storage-driver-user="root" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952474 5030 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952477 5030 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952481 5030 flags.go:64] FLAG: --system-cgroups="" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952483 5030 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952489 5030 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952493 5030 flags.go:64] FLAG: --tls-cert-file="" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952496 5030 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952500 5030 flags.go:64] FLAG: --tls-min-version="" Oct 29 22:33:22.965593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952503 5030 flags.go:64] FLAG: --tls-private-key-file="" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952506 5030 flags.go:64] FLAG: --topology-manager-policy="none" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952508 5030 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952511 5030 flags.go:64] FLAG: --topology-manager-scope="container" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952515 5030 flags.go:64] FLAG: --v="2" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952519 5030 flags.go:64] FLAG: --version="false" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952523 5030 flags.go:64] FLAG: --vmodule="" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952527 5030 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.952530 5030 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952671 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952676 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952680 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952683 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952686 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952689 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952691 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952694 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952697 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952699 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952703 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952706 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.966348 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952708 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952711 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952713 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952716 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952718 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952721 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952723 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952726 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952728 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952731 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952734 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952737 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952739 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952742 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952745 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952748 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952751 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952753 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952756 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952759 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.966965 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952761 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952764 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952766 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952769 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952771 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952774 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952776 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952779 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952781 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952784 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952791 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952793 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952796 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952798 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952801 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952804 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952806 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952809 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952811 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952814 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.967500 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952817 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952819 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952822 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952825 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952827 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952830 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952832 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952848 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952850 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952853 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952856 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952858 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952861 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952864 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952866 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952869 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952871 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952874 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952876 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.968123 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952879 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952881 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952884 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952887 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952890 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952893 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952896 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952898 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952902 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952906 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952910 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952913 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952915 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952918 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952921 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952923 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952926 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952929 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952932 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.968673 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952934 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952938 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952941 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952943 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952946 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952948 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952951 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952953 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952956 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952959 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952961 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952964 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952966 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952969 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952971 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.952974 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.969241 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.953002 5030 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953039 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953044 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953048 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953051 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953054 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953057 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953059 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953062 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953064 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953066 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953069 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953071 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953074 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953077 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953079 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953082 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953084 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953087 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953090 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.969682 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953092 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953095 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953097 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953100 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953102 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953105 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953107 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953110 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953112 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953115 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953117 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953120 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953123 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953126 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953128 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953131 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953133 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953136 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953138 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953141 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.970188 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953143 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953146 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953148 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953151 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953153 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953156 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953158 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953188 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953354 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953365 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953378 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953386 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953394 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953401 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953408 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953416 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953422 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953430 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953436 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953445 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.970699 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953452 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953466 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953473 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953480 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953491 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953498 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953505 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953512 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953520 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953527 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953534 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953542 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953549 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953557 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953570 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953577 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953585 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953592 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953598 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.971237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953605 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953621 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953632 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953642 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953650 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953658 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953672 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953678 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953686 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953692 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953699 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953709 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953716 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953723 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953731 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953738 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953745 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953757 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953769 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.971893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953776 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953782 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953786 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953790 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953794 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953799 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953803 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953807 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.953810 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.958714 5030 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.958732 5030 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.958756 5030 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.958764 5030 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958871 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958879 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958885 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958890 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958897 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958903 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.972588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958907 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958912 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958916 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958920 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958925 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958929 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958933 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958938 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958942 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958947 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958951 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958955 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958959 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958964 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958968 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958972 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958977 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958982 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958986 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.973129 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958991 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958995 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.958999 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959004 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959008 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959013 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959018 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959022 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959027 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959031 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959035 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959040 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959044 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959048 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959052 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959056 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959060 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959065 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959069 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.973612 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959073 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959077 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959082 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959086 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959091 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959095 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959100 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959107 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959111 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959115 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959120 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959124 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959128 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959133 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959138 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959142 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959146 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959150 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959155 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959159 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.974259 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959163 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959168 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959172 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959176 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959180 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959185 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959189 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959193 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959198 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959202 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959206 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959210 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959214 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959219 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959223 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959228 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959233 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959237 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959241 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959246 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.974764 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959252 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959256 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959262 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959266 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959271 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959275 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959279 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959282 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959287 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959291 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959295 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959299 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959303 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959310 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959316 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959321 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959325 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959330 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959334 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.975268 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959338 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959342 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959347 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.959388 5030 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959631 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959641 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959646 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959651 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959655 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959659 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959664 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959668 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959674 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959679 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959684 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959689 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.975716 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959693 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959697 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959702 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959706 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959710 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959715 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959720 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959724 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959728 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959733 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959737 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959741 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959746 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959750 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959755 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959759 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959763 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959767 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959771 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959775 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.976264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959780 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959784 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959789 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959793 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959797 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959801 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959805 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959810 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959813 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959818 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959825 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959832 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959855 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959859 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959863 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959868 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959872 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959876 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959880 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.976810 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959884 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959888 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959893 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959897 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959901 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959906 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959934 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959942 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959948 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959954 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959959 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959963 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959967 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959972 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959976 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959981 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959985 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959989 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959993 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.959997 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.977443 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960001 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960006 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960010 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960014 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960018 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960024 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960028 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960032 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960036 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960040 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960044 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960048 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960053 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960057 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960062 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960066 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960071 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960076 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960080 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.978108 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960084 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960088 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960094 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960101 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960106 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960111 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960115 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960120 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960124 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960128 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960132 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960137 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960141 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960145 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960149 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960154 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.978644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.960193 5030 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960242 5030 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960249 5030 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960254 5030 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960259 5030 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960264 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960268 5030 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960272 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960276 5030 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960283 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960288 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960292 5030 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960296 5030 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960300 5030 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960304 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960309 5030 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960313 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960318 5030 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960322 5030 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960326 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 22:33:22.979069 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960330 5030 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960334 5030 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960338 5030 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960342 5030 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960347 5030 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960351 5030 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960356 5030 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960360 5030 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960364 5030 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960368 5030 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960373 5030 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960377 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960381 5030 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960385 5030 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960389 5030 feature_gate.go:386] unrecognized feature gate: Example Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960396 5030 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960404 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960409 5030 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960413 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 22:33:22.979588 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960417 5030 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960422 5030 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960428 5030 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960432 5030 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960436 5030 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960440 5030 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960445 5030 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960449 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960453 5030 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960457 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960461 5030 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960465 5030 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960470 5030 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960474 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960478 5030 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960483 5030 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960487 5030 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960491 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960495 5030 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 22:33:22.980055 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960500 5030 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960504 5030 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960509 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960513 5030 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960517 5030 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960521 5030 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960525 5030 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960529 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960533 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960538 5030 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960542 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960547 5030 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960551 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960555 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960559 5030 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960565 5030 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960570 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960574 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960578 5030 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 22:33:22.980685 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960582 5030 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960586 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960590 5030 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960596 5030 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960600 5030 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960604 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960608 5030 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960613 5030 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960619 5030 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960625 5030 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960629 5030 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960634 5030 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960638 5030 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960642 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960646 5030 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960651 5030 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960655 5030 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960660 5030 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960664 5030 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960668 5030 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 22:33:22.981242 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960672 5030 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960677 5030 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960681 5030 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960685 5030 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960689 5030 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960695 5030 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960699 5030 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960704 5030 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960709 5030 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:22.960714 5030 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.960786 5030 server.go:957] "Client rotation is on, will bootstrap in background" Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.962306 5030 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.962378 5030 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.962636 5030 server.go:1014] "Starting client certificate rotation" Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.962777 5030 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.962826 5030 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 22:27:42 +0000 UTC" deadline="2028-04-12 17:21:12.741150078 +0000 UTC" Oct 29 22:33:22.981729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.962878 5030 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12738h47m49.778274076s" Oct 29 22:33:22.982330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.963367 5030 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 22:33:22.982330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.965175 5030 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 22:33:22.982330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.966560 5030 log.go:25] "Validated CRI v1 runtime API" Oct 29 22:33:22.982330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.968324 5030 log.go:25] "Validated CRI v1 image API" Oct 29 22:33:22.982330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.969094 5030 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 29 22:33:22.982330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.971805 5030 fs.go:135] Filesystem UUIDs: map[71ddfd82-bcc0-45fc-8bf2-2467e103f872:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f65ffe2d-f7db-4d59-86ed-2696b48b2ec8:/dev/nvme0n1p4] Oct 29 22:33:22.982624 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.971873 5030 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51/userdata/shm major:0 minor:352 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/243f4843e6aedded216f69985397aad912f0de6e9364f4f1269c8cfbf0beea36/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/243f4843e6aedded216f69985397aad912f0de6e9364f4f1269c8cfbf0beea36/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/43ab2caceddd95f7c63da7115cfb15ff9df0fb3216ffb913aa195b53728d2888/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/43ab2caceddd95f7c63da7115cfb15ff9df0fb3216ffb913aa195b53728d2888/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b/userdata/shm major:0 minor:301 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465/userdata/shm major:0 minor:338 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~projected/kube-api-access-vmsj6:{mountpoint:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~projected/kube-api-access-vmsj6 major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volumes/kubernetes.io~projected/kube-api-access-zzfch:{mountpoint:/var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volumes/kubernetes.io~projected/kube-api-access-zzfch major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ac5c5e2-a666-4cbf-9a73-677929dc640e/volumes/kubernetes.io~projected/kube-api-access-nmq6n:{mountpoint:/var/lib/kubelet/pods/1ac5c5e2-a666-4cbf-9a73-677929dc640e/volumes/kubernetes.io~projected/kube-api-access-nmq6n major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/230e7bac-638f-4478-989a-abad8f7213bd/volumes/kubernetes.io~projected/kube-api-access-4ghgf:{mountpoint:/var/lib/kubelet/pods/230e7bac-638f-4478-989a-abad8f7213bd/volumes/kubernetes.io~projected/kube-api-access-4ghgf major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24316779-cb78-42df-954b-c36d2b0e097d/volumes/kubernetes.io~projected/kube-api-access-8hkth:{mountpoint:/var/lib/kubelet/pods/24316779-cb78-42df-954b-c36d2b0e097d/volumes/kubernetes.io~projected/kube-api-access-8hkth major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24348922-a2e2-4e39-9166-1aed0076055c/volumes/kubernetes.io~projected/kube-api-access-7rx5d:{mountpoint:/var/lib/kubelet/pods/24348922-a2e2-4e39-9166-1aed0076055c/volumes/kubernetes.io~projected/kube-api-access-7rx5d major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~projected/kube-api-access-7mrrj:{mountpoint:/var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~projected/kube-api-access-7mrrj major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019/volumes/kubernetes.io~projected/kube-api-access-6bzv6:{mountpoint:/var/lib/kubelet/pods/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019/volumes/kubernetes.io~projected/kube-api-access-6bzv6 major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3374d1d7-08e1-409a-8361-cf995598345b/volumes/kubernetes.io~projected/kube-api-access-zsb72:{mountpoint:/var/lib/kubelet/pods/3374d1d7-08e1-409a-8361-cf995598345b/volumes/kubernetes.io~projected/kube-api-access-zsb72 major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f37d2fc-3cf7-4105-9c73-bf532a71af91/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/3f37d2fc-3cf7-4105-9c73-bf532a71af91/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20/volumes/kubernetes.io~projected/kube-api-access-zpmqc:{mountpoint:/var/lib/kubelet/pods/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20/volumes/kubernetes.io~projected/kube-api-access-zpmqc major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8/volumes/kubernetes.io~projected/kube-api-access-vw9pj:{mountpoint:/var/lib/kubelet/pods/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8/volumes/kubernetes.io~projected/kube-api-access-vw9pj major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ec29132-5b42-4a06-be85-de3789f00145/volumes/kubernetes.io~projected/kube-api-access-lw8wf:{mountpoint:/var/lib/kubelet/pods/4ec29132-5b42-4a06-be85-de3789f00145/volumes/kubernetes.io~projected/kube-api-access-lw8wf major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ec29132-5b42-4a06-be85-de3789f00145/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/4ec29132-5b42-4a06-be85-de3789f00145/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fa778a3-926f-46e5-8a16-976b7f9819ca/volumes/kubernetes.io~projected/kube-api-access-nk8cd:{mountpoint:/var/lib/kubelet/pods/4fa778a3-926f-46e5-8a16-976b7f9819ca/volumes/kubernetes.io~projected/kube-api-access-nk8cd major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5332e2aa-b433-47d6-b3c3-d42e66376eca/volumes/kubernetes.io~projected/kube-api-access-mkht6:{mountpoint:/var/lib/kubelet/pods/5332e2aa-b433-47d6-b3c3-d42e66376eca/volumes/kubernetes.io~projected/kube-api-access-mkht6 major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7/volumes/kubernetes.io~projected/kube-api-access-5qpwm:{mountpoint:/var/lib/kubelet/pods/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7/volumes/kubernetes.io~projected/kube-api-access-5qpwm major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/volumes/kubernetes.io~projected/kube-api-access-gnxbw:{mountpoint:/var/lib/kubelet/pods/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/volumes/kubernetes.io~projected/kube-api-access-gnxbw major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/64f11246-8026-4e31-b461-1768145983a5/volumes/kubernetes.io~projected/kube-api-access-hwvg8:{mountpoint:/var/lib/kubelet/pods/64f11246-8026-4e31-b461-1768145983a5/volumes/kubernetes.io~projected/kube-api-access-hwvg8 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9/volumes/kubernetes.io~projected/kube-api-access-zwxrx:{mountpoint:/var/lib/kubelet/pods/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9/volumes/kubernetes.io~projected/kube-api-access-zwxrx major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8978b075-27a6-4c34-b4d4-36270f768a01/volumes/kubernetes.io~projected/kube-api-access-nvcvp:{mountpoint:/var/lib/kubelet/pods/8978b075-27a6-4c34-b4d4-36270f768a01/volumes/kubernetes.io~projected/kube-api-access-nvcvp major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8db65a63-2853-43db-8877-83a0d26c2de2/volumes/kubernetes.io~projected/kube-api-access-wqblx:{mountpoint:/var/lib/kubelet/pods/8db65a63-2853-43db-8877-83a0d26c2de2/volumes/kubernetes.io~projected/kube-api-access-wqblx major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8db65a63-2853-43db-8877-83a0d26c2de2/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/8db65a63-2853-43db-8877-83a0d26c2de2/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/901d2e7b-558b-4fc4-833b-a71699d02494/volumes/kubernetes.io~projected/kube-api-access-bbrmd:{mountpoint:/var/lib/kubelet/pods/901d2e7b-558b-4fc4-833b-a71699d02494/volumes/kubernetes.io~projected/kube-api-access-bbrmd major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/983a2044-473f-48be-8e73-5da2ed82aaf4/volumes/kubernetes.io~projected/kube-api-access-bpjjq:{mountpoint:/var/lib/kubelet/pods/983a2044-473f-48be-8e73-5da2ed82aaf4/volumes/kubernetes.io~projected/kube-api-access-bpjjq major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032/volumes/kubernetes.io~projected/kube-api-access-25p5w:{mountpoint:/var/lib/kubelet/pods/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032/volumes/kubernetes.io~projected/kube-api-access-25p5w major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8/volumes/kubernetes.io~projected/kube-api-access-g8wxt:{mountpoint:/var/lib/kubelet/pods/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8/volumes/kubernetes.io~projected/kube-api-access-g8wxt major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c7881bd2-9f24-4c58-9a77-68cc091ee9ab/volumes/kubernetes.io~projected/kube-api-access-557gg:{mountpoint:/var/lib/kubelet/pods/c7881bd2-9f24-4c58-9a77-68cc091ee9ab/volumes/kubernetes.io~projected/kube-api-access-557gg major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8b45469-5973-49bc-b8c2-8ec0dce0e242/volumes/kubernetes.io~projected/kube-api-access-q87h4:{mountpoint:/var/lib/kubelet/pods/d8b45469-5973-49bc-b8c2-8ec0dce0e242/volumes/kubernetes.io~projected/kube-api-access-q87h4 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8b45469-5973-49bc-b8c2-8ec0dce0e242/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d8b45469-5973-49bc-b8c2-8ec0dce0e242/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e5092efe-96fd-4aec-99f1-a2c07c84ac7f/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/e5092efe-96fd-4aec-99f1-a2c07c84ac7f/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~projected/kube-api-access-vb729:{mountpoint:/var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~projected/kube-api-access-vb729 major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~secret/default-certificate major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~secret/stats-auth major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed3a898c-87c6-409d-a70f-f01e8e15d787/volumes/kubernetes.io~projected/kube-api-access-khhs6:{mountpoint:/var/lib/kubelet/pods/ed3a898c-87c6-409d-a70f-f01e8e15d787/volumes/kubernetes.io~projected/kube-api-access-khhs6 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed3a898c-87c6-409d-a70f-f01e8e15d787/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/ed3a898c-87c6-409d-a70f-f01e8e15d787/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~projected/kube-api-access-8nfrs:{mountpoint:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~projected/kube-api-access-8nfrs major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/efa7fc2d-d350-4aba-977b-32868a6e3fe7/volumes/kubernetes.io~projected/kube-api-access-8t79s:{mountpoint:/var/lib/kubelet/pods/efa7fc2d-d350-4aba-977b-32868a6e3fe7/volumes/kubernetes.io~projected/kube-api-access-8t79s major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fffc6a30-b103-428e-a73e-7d7d03a4e2ff/volumes/kubernetes.io~projected/kube-api-access-cxs2w:{mountpoint:/var/lib/kubelet/pods/fffc6a30-b103-428e-a73e-7d7d03a4e2ff/volumes/kubernetes.io~projected/kube-api-access-cxs2w major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fffc6a30-b103-428e-a73e-7d7d03a4e2ff/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/fffc6a30-b103-428e-a73e-7d7d03a4e2ff/volumes/kubernetes.io~secret/signing-key major:0 minor:346 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/d80881dfefd687bdbe5fe26c415991a4460bb1c756ee1fd2879b1ca9b0562d2b/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/0abdfbd02debc83dd6eabaefba0ed14d5405d1d6365b42fbeea9a3ca751a108e/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/c158231f50c5655dd794381b5a82c321e484e4ea7519c3e059e478ac7352ed61/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/17164676a40f5383eb91385ee06d06ade72693af506fff653836a0ce0b1f329c/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/a53ee27dbfa4762b9d0165bcb3f7a8b60fcae423cc96b773b1e884d89d0b319d/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/1fb87696091a54d61680f9a0460ab967f6ef5c35bc863b74c4088db3aa726c22/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/442391c93fdf7c0c89fa03b0ebda893e8f6a946ee4f183fa5db5f017aee908c4/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/8da0910a6c733f77254496b276b64d53c18aa27ba3fe4bc4f529df4af16202ab/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/b6dc2e83b2b73e324505de99b17e6a60ab5fc5298856dc97e80397d7730e2252/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/70f08475f46f3cf8639d16fea6e0e16232ad5118a67cce9095035a147455a972/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/37f30fb78579689987f72845c6f220f66148c6b6f31c3f1277f9f3de2a08dc4e/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/d198676676cc38936de36c2282ac5d75e95808f22c461522f541ba9c8b59602c/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/e77a60245aca6b55b39b51244ec137abbbaafd08ac0289f286c2b2c7492f91c3/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/506d16d980706ce8dad27a2cb7771299565d05a344a5e99aca25963ce1c1bad6/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f2061b0d7064364599046a13b0c8447c6735a83ad2c002f251abb616a72ffdd3/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/b8cb7fcc366502920f3c37d50bf3fc26dcd25ae709f90fa9470ad0880bb2b564/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-173:{mountpoint:/var/lib/containers/storage/overlay/45f68c6ee523357409dbc590438a1f23f1713d95b06621354e7a305547779616/merged major:0 minor:173 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/842e0c8df009fb24881701ab8a9a75224027f109303cdd39e9ce23b3de83518f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/bd28b7c0d8cae77867b4f40a6462fed4b329beca4fa1325281f1fc5b2a0d97be/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/56e519528b51ef4cec12e369abd25014908f57bc096ceadd7eeb4431173ffdf1/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/eb9c52ff288dc71bc3c11d8e7db5b7f7e97c0c6bfb42b4e85890f344f1c12fd7/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/1ceb98d590c680048f273712b4860ba72e5d57971d8f9d99c3eaad8a813ea640/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/aaebe35add706d60cb767e5e8572d40a01c75add25fb173809e887fd57f47e53/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/b4d2d908e125593f32c999554ee7430103d1fad12da0d174ff93f4cc2634f30e/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/c12559fdf5790bfb0580c0a9a6db3819d27b476c0a847268a28c56e5a386f898/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/58c1f1a646eb5f2fd558a918490b6601f4acfd5ba29bd09172f3d9f92f2429e3/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/cc08d40635dd6be300dc2ad2b510286d29fd4f982e68080b9e46078488d8af23/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/e5f9297ee2f466f1ea89d1bea4730f7ebf7f64eff415b03176acd24f36410d57/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/da58b925982d39178d97184ab2b1b5c6abf9021299312a3562a19d2f2552963b/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/edd2d0181bc7641458c81d4ad03d99e1fcf526a922a872338fde551cd04d9d94/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/c380485776ba15e699edcb207513d07f55330d6ea8bd96ec7a45752e6228e2c5/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/688c25dec79354dc9422afd58918bb05ee49da8a8a0a5278a68dd5662bef4f74/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/26aef72a399a6bf2a387c25ea4655362523d3c7ee1b7f91a410284e39626cece/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/9abbd765dc58429986b6a7d8b7706c7b2ee7645b8949ec87a1aec063bc2f3c7b/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/402c2d645a2e9259c7423df1701e83febd9e664c8e7644a7329711a32928e28e/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/bd243208e4c796f76dd3587b50f24a77559fb4b1ff4470f84fcbaed1d782183c/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-297:{mountpoint:/var/lib/containers/storage/overlay/ab07cd613f22ac5c93d221169e1dcda679bc9fd76229f873cdf6f8702f47eb70/merged major:0 minor:297 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/33b9afda92db7c0ce17fa9cf0d9fc655b145e06e2d79f88bc63b68f1bee221de/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/a0060fd3bf752c92db3a8761883a3d9206a57da6132c4c114bd9ef4259beb16c/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/6ec74d5fef99e7b18f48cc4c37301842d3c340d196fa998a29026efc2e6d22e6/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/6cd365a7030ef0d09f5cd211aed08299f165977c74e35a0e19d5b75054b3c381/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/388c66c2adb9a97d031a5803f323d87f8196a3d38a9eb98a699024d0898e9f9b/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/21676225042d80975238f760754f6966c4805c95a290f5104991ce340bfcf379/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/184de742509730f8c20975c2cbad9bd6031760e83399ab0401d419ff5f2e70f2/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/992b66f49edce9b148f462d0671e1927d3103560cecc73e409bf55087dcc08a9/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/c6a243fa117e3a693fc0469fbace4715df3b3d8ee357c2fbdb191163e342ac1d/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/aaf40a9f1831b6a4c194d70fc2f0ea5338d72d156210355d538610b073638ddb/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/433d669b0058b0f5416f1f7586bfa11bf12f28671fe043268282cd3f7245f8f1/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/dae44549132bba681b182484475b49314086dbfa47efba6f0612ebc0c920a45f/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/aea61d2e61110ada1646657c92de222edfa4d6276cfe5e2968ecb31946fcf659/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/f39a34db451dfd7b90d181d62832db06a8076774e5315d4e4a17e3b47935f3df/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/d387fe547f1287b2436d0cb7b82075ca8a7c59c848902b9965a4c1664568f9b8/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/587f2e733520d9db034fb97b1c49b3a2c48cea756350f33eac80c2bd00852dac/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/a5b7072180c4da0550db580cde4b408e20f256f7202bc47562816f08eccb7f25/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ed1e52f4d57d5d1403fbf0ae134074660374dcdb0ba9f74f873e73a220a583ce/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/01a17b7103c83e8e804167d5bac770b96c441a5200219a0ccd97564207fa6e93/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/4dd55bacb87c90e8d46d8326691f8b7d785be6beab207ed275444151a38b7442/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/af365ee314702d024e478deece125a35610d17b721efd25e644c5be428191a81/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/24991114ea96f5290e274d5268897b473252e7061bc717e0bea8c5a0db7ecc97/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/de72bc48510fec242be1279ba983e785102807f36b30734af08f8ca7422c6a7f/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/6f6a79c3eabbc5d1cde455e8cfa2e8998bcc5916064ad5462991c05e1c673d6e/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/209fdd48764b481861eee0e72134a40305439ddf43f69409249f9b9622ed0495/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.980608 5030 manager.go:217] Machine: {Timestamp:2026-10-29 22:33:22.98016716 +0000 UTC m=+0.092812739 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100778 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec230f13f49fac39b0f1879e548ac801 SystemUUID:ec230f13-f49f-ac39-b0f1-879e548ac801 BootID:54af53cc-5e97-46bd-b7ce-40762eb8adfd Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d82037025180716a4a8640a8a8ac1e529ae9dc2711c2fa70c6ffebc1d208e905/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032/volumes/kubernetes.io~projected/kube-api-access-25p5w DeviceMajor:0 DeviceMinor:293 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/49131b91142c2302330eb9cb4fe6b5bbe615ec6d72d8c5d7ef76e12a2ef19de7/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/507144ebeeb8fa9a7edf6cba5b987f9d7a4e1de367c9f074abad36c32297f98d/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:208 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c9e195e9a3d10db141e80e0cf3649abcad638d38d29de623dd4d9ebd3de6a246/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:205 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ed3a898c-87c6-409d-a70f-f01e8e15d787/volumes/kubernetes.io~projected/kube-api-access-khhs6 DeviceMajor:0 DeviceMinor:224 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7b441425af5f96fd33902f9e6d44cbc85c5e9460bc619a7c37340d9a36ef933f/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/65ecd755198b8a2be573989997403d3eb7628c508b7b9fd5b66796352fa4d9d2/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9/volumes/kubernetes.io~projected/kube-api-access-zwxrx DeviceMajor:0 DeviceMinor:221 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e60bb7cfcbc120bfd149590aa25e5e17ba96f0c01eb65f8527407e73448be37c/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5b9285f1dbb64d93b9715e96ecc7982caecebe061225c3f0d1820c6242928fb7/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/da46f48306cfe46b1f7369a0508a904ad1eb5a30bafb874f108e689330dc3109/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~projected/kube-api-access-7mrrj DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volumes/kubernetes.io~projected/kube-api-access-zzfch DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/39431137a80ddf4f8e122cea2ab4c0408729f58937665b62d2ca7f4be2f7fe91/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b72fbe404bcc13a08462313454e026bac86501b8b7ca7413ed8bf07693f163aa/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fffc6a30-b103-428e-a73e-7d7d03a4e2ff/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:346 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4ec29132-5b42-4a06-be85-de3789f00145/volumes/kubernetes.io~projected/kube-api-access-lw8wf DeviceMajor:0 DeviceMinor:227 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c7881bd2-9f24-4c58-9a77-68cc091ee9ab/volumes/kubernetes.io~projected/kube-api-access-557gg DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:206 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/230e7bac-638f-4478-989a-abad8f7213bd/volumes/kubernetes.io~projected/kube-api-access-4ghgf DeviceMajor:0 DeviceMinor:211 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/243f4843e6aedded216f69985397aad912f0de6e9364f4f1269c8cfbf0beea36/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed3a898c-87c6-409d-a70f-f01e8e15d787/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/983a2044-473f-48be-8e73-5da2ed82aaf4/volumes/kubernetes.io~projected/kube-api-access-bpjjq DeviceMajor:0 DeviceMinor:213 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/efa7fc2d-d350-4aba-977b-32868a6e3fe7/volumes/kubernetes.io~projected/kube-api-access-8t79s DeviceMajor:0 DeviceMinor:218 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/84e3a600d2921689a8720b50bcea59ce232babdf32aa1970b8bc1c7f2e093b9b/userdata/shm DeviceMajor:0 DeviceMinor:301 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8db65a63-2853-43db-8877-83a0d26c2de2/volumes/kubernetes.io~projected/kube-api-access-wqblx DeviceMajor:0 DeviceMinor:210 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4ec29132-5b42-4a06-be85-de3789f00145/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e5092efe-96fd-4aec-99f1-a2c07c84ac7f/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/19cc851c85fb6051ed819026d3cd96afcf7d0ba60511a43f219ac5342bd8fd5b/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-173 DeviceMajor:0 DeviceMinor:173 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:215 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019/volumes/kubernetes.io~projected/kube-api-access-6bzv6 DeviceMajor:0 DeviceMinor:216 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/901d2e7b-558b-4fc4-833b-a71699d02494/volumes/kubernetes.io~projected/kube-api-access-bbrmd DeviceMajor:0 DeviceMinor:325 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/28c8a747-4790-4ff6-9c11-a1b66d5a98c3/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/64f11246-8026-4e31-b461-1768145983a5/volumes/kubernetes.io~projected/kube-api-access-hwvg8 DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/24316779-cb78-42df-954b-c36d2b0e097d/volumes/kubernetes.io~projected/kube-api-access-8hkth DeviceMajor:0 DeviceMinor:225 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3117859de41edaf5da6d677426a1ca5f089547dc5836e1a4a8b0a9a6238aef1e/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff8181abe15b2e2579745ec7d6027220fcd5d7d6b46cfcce94b6bb2307669132/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/deb6e2c80d0148eb8502b7616bb08ccbf03f8a0e132b8bdcc95aa1b6cebe78e8/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/18f38e0d005d85e799f723423d84a24baafda5498d1a4f261368a693580a7827/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fffc6a30-b103-428e-a73e-7d7d03a4e2ff/volumes/kubernetes.io~projected/kube-api-access-cxs2w DeviceMajor:0 DeviceMinor:347 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:220 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d8b45469-5973-49bc-b8c2-8ec0dce0e242/volumes/kubernetes.io~projected/kube-api-access-q87h4 DeviceMajor:0 DeviceMinor:222 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8978b075-27a6-4c34-b4d4-36270f768a01/volumes/kubernetes.io~projected/kube-api-access-nvcvp DeviceMajor:0 DeviceMinor:324 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00cdb36c1140ebd0181740bcdec3f6c480a961ee3ef13674ade1ed4f87c2cb51/userdata/shm DeviceMajor:0 DeviceMinor:352 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:209 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4fa778a3-926f-46e5-8a16-976b7f9819ca/volumes/kubernetes.io~projected/kube-api-access-nk8cd DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ac5c5e2-a666-4cbf-9a73-677929dc640e/volumes/kubernetes.io~projected/kube-api-access-nmq6n DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20/volumes/kubernetes.io~projected/kube-api-access-zpmqc DeviceMajor:0 DeviceMinor:214 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5332e2aa-b433-47d6-b3c3-d42e66376eca/volumes/kubernetes.io~projected/kube-api-access-mkht6 DeviceMajor:0 DeviceMinor:217 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e9fb3bd9-7312-4616-a8db-6e51b56d78a2/volumes/kubernetes.io~projected/kube-api-access-vb729 DeviceMajor:0 DeviceMinor:226 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/43ab2caceddd95f7c63da7115cfb15ff9df0fb3216ffb913aa195b53728d2888/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3374d1d7-08e1-409a-8361-cf995598345b/volumes/kubernetes.io~projected/kube-api-access-zsb72 DeviceMajor:0 DeviceMinor:334 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82e89061f1ed9bd66593c84449df97553815d619383a0bbc213e64a40c7b64c5/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24348922-a2e2-4e39-9166-1aed0076055c/volumes/kubernetes.io~projected/kube-api-access-7rx5d DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes/kubernetes.io~projected/kube-api-access-vmsj6 DeviceMajor:0 DeviceMinor:223 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/da707ce273910cee5652f206fcf33f756199718ab5f17c8385982de8269d6465/userdata/shm DeviceMajor:0 DeviceMinor:338 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/volumes/kubernetes.io~projected/kube-api-access-gnxbw DeviceMajor:0 DeviceMinor:212 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f58ef82e62c927fb2bda88589ea5b165346249b161b9f05d3bf6f6cd60fd44b3/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8db65a63-2853-43db-8877-83a0d26c2de2/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes/kubernetes.io~projected/kube-api-access-8nfrs DeviceMajor:0 DeviceMinor:219 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-297 DeviceMajor:0 DeviceMinor:297 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7/volumes/kubernetes.io~projected/kube-api-access-5qpwm DeviceMajor:0 DeviceMinor:335 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d8b45469-5973-49bc-b8c2-8ec0dce0e242/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8/volumes/kubernetes.io~projected/kube-api-access-g8wxt DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19563c05-8911-48a9-bdf0-ca368119b49d/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8/volumes/kubernetes.io~projected/kube-api-access-vw9pj DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f37d2fc-3cf7-4105-9c73-bf532a71af91/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:00cdb36c1140ebd MacAddress:f6:3e:0d:83:5f:60 Speed:10000 Mtu:8901} {Name:18f38e0d005d85e MacAddress:7e:b7:24:8c:b1:62 Speed:10000 Mtu:8901} {Name:243f4843e6aedde MacAddress:f2:f1:b7:ac:07:39 Speed:10000 Mtu:8901} {Name:3117859de41edaf MacAddress:22:ed:64:42:cf:8b Speed:10000 Mtu:8901} {Name:43ab2caceddd95f MacAddress:2a:8a:b2:33:00:40 Speed:10000 Mtu:8901} {Name:5b9285f1dbb64d9 MacAddress:d2:4c:d9:9a:ff:61 Speed:10000 Mtu:8901} {Name:7b441425af5f96f MacAddress:ce:bf:40:1b:ba:17 Speed:10000 Mtu:8901} {Name:82e89061f1ed9bd MacAddress:72:c8:9c:ee:5c:e4 Speed:10000 Mtu:8901} {Name:84e3a600d292168 MacAddress:46:78:65:2b:3c:c5 Speed:10000 Mtu:8901} {Name:b72fbe404bcc13a MacAddress:ee:12:89:9f:70:18 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c8:08:ac:cf Speed:0 Mtu:9001} {Name:br-int MacAddress:b2:6d:57:fb:82:fe Speed:0 Mtu:8901} {Name:c9e195e9a3d10db MacAddress:3a:39:dc:db:1f:35 Speed:10000 Mtu:8901} {Name:da46f48306cfe46 MacAddress:e6:c4:4e:85:ae:4f Speed:10000 Mtu:8901} {Name:da707ce273910ce MacAddress:92:dd:d8:ec:31:b1 Speed:10000 Mtu:8901} {Name:deb6e2c80d0148e MacAddress:32:fc:3c:a7:ad:85 Speed:10000 Mtu:8901} {Name:e6255efd3425fc3 MacAddress:86:1e:2d:1c:61:99 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c8:08:ac:cf Speed:0 Mtu:9001} {Name:f58ef82e62c927f MacAddress:6e:59:a2:e8:df:58 Speed:10000 Mtu:8901} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:da:c6:2b:a9:9c:62 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.980993 5030 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981031 5030 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981308 5030 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981328 5030 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-228.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981449 5030 topology_manager.go:143] "Creating topology manager with none policy" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981455 5030 container_manager_linux.go:311] "Creating device plugin manager" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981466 5030 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981478 5030 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981561 5030 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981706 5030 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981797 5030 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981830 5030 kubelet.go:493] "Attempting to sync node with API server" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981858 5030 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981872 5030 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981883 5030 kubelet.go:405] "Adding apiserver pod source" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.981890 5030 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.982634 5030 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 29 22:33:22.983035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.982644 5030 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 22:33:22.984173 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.983063 5030 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 29 22:33:22.985569 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985551 5030 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 29 22:33:22.985569 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985565 5030 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 22:33:22.985691 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985659 5030 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 29 22:33:22.985852 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985826 5030 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 29 22:33:22.985887 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985877 5030 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 29 22:33:22.985971 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985962 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985976 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985982 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985988 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985993 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.985999 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986005 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986010 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 29 22:33:22.986014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986016 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 29 22:33:22.986333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986021 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 29 22:33:22.986333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986036 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 29 22:33:22.986333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986046 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 29 22:33:22.986333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986071 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 29 22:33:22.986333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.986077 5030 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 29 22:33:22.989396 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.989378 5030 server.go:1263] "Started kubelet" Oct 29 22:33:22.990064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.990033 5030 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 29 22:33:22.992754 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.990328 5030 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 29 22:33:22.992754 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.990407 5030 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 29 22:33:22.992754 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.990816 5030 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 29 22:33:22.992754 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.992290 5030 server.go:317] "Adding debug handlers to kubelet server" Oct 29 22:33:22.990775 ip-10-0-136-228 systemd[1]: Started Kubernetes Kubelet. Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.996675 5030 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.996693 5030 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.996809 5030 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.996821 5030 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:22.996855 5030 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-228.ec2.internal\" not found" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.996865 5030 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 22:27:44 +0000 UTC" deadline="2028-06-05 11:05:10.931627004 +0000 UTC" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.996896 5030 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14028h31m47.934733762s" Oct 29 22:33:22.999306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:22.997020 5030 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 29 22:33:23.000128 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000097 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" volumeName="kubernetes.io/projected/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-kube-api-access-g8wxt" seLinuxMountContext="" Oct 29 22:33:23.000222 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000204 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration" seLinuxMountContext="" Oct 29 22:33:23.000286 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000224 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ac5c5e2-a666-4cbf-9a73-677929dc640e" volumeName="kubernetes.io/empty-dir/1ac5c5e2-a666-4cbf-9a73-677929dc640e-tmp-dir" seLinuxMountContext="" Oct 29 22:33:23.000286 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000247 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3374d1d7-08e1-409a-8361-cf995598345b" volumeName="kubernetes.io/projected/3374d1d7-08e1-409a-8361-cf995598345b-kube-api-access-zsb72" seLinuxMountContext="" Oct 29 22:33:23.000286 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000261 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" volumeName="kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle" seLinuxMountContext="" Oct 29 22:33:23.000286 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000273 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" volumeName="kubernetes.io/projected/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-kube-api-access-gnxbw" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000285 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="901d2e7b-558b-4fc4-833b-a71699d02494" volumeName="kubernetes.io/projected/901d2e7b-558b-4fc4-833b-a71699d02494-kube-api-access-bbrmd" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000298 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3a898c-87c6-409d-a70f-f01e8e15d787" volumeName="kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-service-ca-bundle" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000314 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000325 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000338 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28c8a747-4790-4ff6-9c11-a1b66d5a98c3" volumeName="kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-tuned" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000350 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" volumeName="kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000361 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8978b075-27a6-4c34-b4d4-36270f768a01" volumeName="kubernetes.io/projected/8978b075-27a6-4c34-b4d4-36270f768a01-kube-api-access-nvcvp" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000381 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db65a63-2853-43db-8877-83a0d26c2de2" volumeName="kubernetes.io/projected/8db65a63-2853-43db-8877-83a0d26c2de2-kube-api-access-wqblx" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000396 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db65a63-2853-43db-8877-83a0d26c2de2" volumeName="kubernetes.io/secret/8db65a63-2853-43db-8877-83a0d26c2de2-cert" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000407 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9fb3bd9-7312-4616-a8db-6e51b56d78a2" volumeName="kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-default-certificate" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000420 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19563c05-8911-48a9-bdf0-ca368119b49d" volumeName="kubernetes.io/projected/19563c05-8911-48a9-bdf0-ca368119b49d-kube-api-access-zzfch" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000432 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce7d6cc-47fa-4a6a-ba52-5a23efd31019" volumeName="kubernetes.io/configmap/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-telemetry-config" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000444 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9" volumeName="kubernetes.io/projected/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9-kube-api-access-zwxrx" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000460 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8b45469-5973-49bc-b8c2-8ec0dce0e242" volumeName="kubernetes.io/projected/d8b45469-5973-49bc-b8c2-8ec0dce0e242-kube-api-access-q87h4" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000472 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3a898c-87c6-409d-a70f-f01e8e15d787" volumeName="kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-snapshots" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000483 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efa7fc2d-d350-4aba-977b-32868a6e3fe7" volumeName="kubernetes.io/configmap/efa7fc2d-d350-4aba-977b-32868a6e3fe7-config-volume" seLinuxMountContext="" Oct 29 22:33:23.000504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000502 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fffc6a30-b103-428e-a73e-7d7d03a4e2ff" volumeName="kubernetes.io/secret/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-key" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000514 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000525 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce7d6cc-47fa-4a6a-ba52-5a23efd31019" volumeName="kubernetes.io/projected/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-kube-api-access-6bzv6" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000537 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3374d1d7-08e1-409a-8361-cf995598345b" volumeName="kubernetes.io/empty-dir/3374d1d7-08e1-409a-8361-cf995598345b-data-volume" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000549 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d6f9752-92e3-4e94-9ac0-a812fc12c7d8" volumeName="kubernetes.io/projected/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-kube-api-access-vw9pj" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000561 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" volumeName="kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000579 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" volumeName="kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-trusted-ca" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000591 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000602 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efa7fc2d-d350-4aba-977b-32868a6e3fe7" volumeName="kubernetes.io/empty-dir/efa7fc2d-d350-4aba-977b-32868a6e3fe7-tmp-dir" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000614 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24316779-cb78-42df-954b-c36d2b0e097d" volumeName="kubernetes.io/projected/24316779-cb78-42df-954b-c36d2b0e097d-kube-api-access-8hkth" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000625 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ec29132-5b42-4a06-be85-de3789f00145" volumeName="kubernetes.io/projected/4ec29132-5b42-4a06-be85-de3789f00145-kube-api-access-lw8wf" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000637 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" volumeName="kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-config" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000651 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bba55700-7326-482d-919f-02f3eff316a1" volumeName="kubernetes.io/configmap/bba55700-7326-482d-919f-02f3eff316a1-nginx-conf" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000662 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3a898c-87c6-409d-a70f-f01e8e15d787" volumeName="kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-trusted-ca-bundle" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000674 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fffc6a30-b103-428e-a73e-7d7d03a4e2ff" volumeName="kubernetes.io/configmap/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-cabundle" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000686 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19563c05-8911-48a9-bdf0-ca368119b49d" volumeName="kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-env-overrides" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000697 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f37d2fc-3cf7-4105-9c73-bf532a71af91" volumeName="kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000710 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fa778a3-926f-46e5-8a16-976b7f9819ca" volumeName="kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-cni-binary-copy" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000722 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="64f11246-8026-4e31-b461-1768145983a5" volumeName="kubernetes.io/projected/64f11246-8026-4e31-b461-1768145983a5-kube-api-access-hwvg8" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000733 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="983a2044-473f-48be-8e73-5da2ed82aaf4" volumeName="kubernetes.io/projected/983a2044-473f-48be-8e73-5da2ed82aaf4-kube-api-access-bpjjq" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000745 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19563c05-8911-48a9-bdf0-ca368119b49d" volumeName="kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-script-lib" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000756 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" volumeName="kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000769 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000779 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5332e2aa-b433-47d6-b3c3-d42e66376eca" volumeName="kubernetes.io/projected/5332e2aa-b433-47d6-b3c3-d42e66376eca-kube-api-access-mkht6" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000792 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" volumeName="kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000804 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" volumeName="kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-binary-copy" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000817 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5092efe-96fd-4aec-99f1-a2c07c84ac7f" volumeName="kubernetes.io/secret/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-agent-certs" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000828 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3a898c-87c6-409d-a70f-f01e8e15d787" volumeName="kubernetes.io/projected/ed3a898c-87c6-409d-a70f-f01e8e15d787-kube-api-access-khhs6" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000877 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000890 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000902 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="230e7bac-638f-4478-989a-abad8f7213bd" volumeName="kubernetes.io/projected/230e7bac-638f-4478-989a-abad8f7213bd-kube-api-access-4ghgf" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000915 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa67ab99-04d4-4f83-a61b-d6f8fe0b0032" volumeName="kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000926 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5092efe-96fd-4aec-99f1-a2c07c84ac7f" volumeName="kubernetes.io/configmap/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-konnectivity-ca" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000939 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000960 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="efa7fc2d-d350-4aba-977b-32868a6e3fe7" volumeName="kubernetes.io/projected/efa7fc2d-d350-4aba-977b-32868a6e3fe7-kube-api-access-8t79s" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000973 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24348922-a2e2-4e39-9166-1aed0076055c" volumeName="kubernetes.io/projected/24348922-a2e2-4e39-9166-1aed0076055c-kube-api-access-7rx5d" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000985 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.000997 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19563c05-8911-48a9-bdf0-ca368119b49d" volumeName="kubernetes.io/secret/19563c05-8911-48a9-bdf0-ca368119b49d-ovn-node-metrics-cert" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001009 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ec29132-5b42-4a06-be85-de3789f00145" volumeName="kubernetes.io/configmap/4ec29132-5b42-4a06-be85-de3789f00145-config" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001021 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" volumeName="kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-sysctl-allowlist" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001033 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7881bd2-9f24-4c58-9a77-68cc091ee9ab" volumeName="kubernetes.io/configmap/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-serviceca" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001045 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9fb3bd9-7312-4616-a8db-6e51b56d78a2" volumeName="kubernetes.io/projected/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-kube-api-access-vb729" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001058 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3a898c-87c6-409d-a70f-f01e8e15d787" volumeName="kubernetes.io/secret/ed3a898c-87c6-409d-a70f-f01e8e15d787-serving-cert" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001070 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001082 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fa778a3-926f-46e5-8a16-976b7f9819ca" volumeName="kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-daemon-config" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001095 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="64f11246-8026-4e31-b461-1768145983a5" volumeName="kubernetes.io/configmap/64f11246-8026-4e31-b461-1768145983a5-iptables-alerter-script" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001106 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8b45469-5973-49bc-b8c2-8ec0dce0e242" volumeName="kubernetes.io/configmap/d8b45469-5973-49bc-b8c2-8ec0dce0e242-config" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001118 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed3a898c-87c6-409d-a70f-f01e8e15d787" volumeName="kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-tmp" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001130 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001143 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ac5c5e2-a666-4cbf-9a73-677929dc640e" volumeName="kubernetes.io/projected/1ac5c5e2-a666-4cbf-9a73-677929dc640e-kube-api-access-nmq6n" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001154 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28c8a747-4790-4ff6-9c11-a1b66d5a98c3" volumeName="kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-tmp" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001165 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20" volumeName="kubernetes.io/projected/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20-kube-api-access-zpmqc" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001178 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ec29132-5b42-4a06-be85-de3789f00145" volumeName="kubernetes.io/secret/4ec29132-5b42-4a06-be85-de3789f00145-serving-cert" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001189 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" volumeName="kubernetes.io/secret/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-serving-cert" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001202 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28c8a747-4790-4ff6-9c11-a1b66d5a98c3" volumeName="kubernetes.io/projected/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-kube-api-access-7mrrj" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001219 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5332e2aa-b433-47d6-b3c3-d42e66376eca" volumeName="kubernetes.io/configmap/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-config" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001240 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8b45469-5973-49bc-b8c2-8ec0dce0e242" volumeName="kubernetes.io/secret/d8b45469-5973-49bc-b8c2-8ec0dce0e242-serving-cert" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001252 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fffc6a30-b103-428e-a73e-7d7d03a4e2ff" volumeName="kubernetes.io/projected/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-kube-api-access-cxs2w" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001263 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" volumeName="kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001276 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c7881bd2-9f24-4c58-9a77-68cc091ee9ab" volumeName="kubernetes.io/projected/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-kube-api-access-557gg" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001288 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9fb3bd9-7312-4616-a8db-6e51b56d78a2" volumeName="kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-stats-auth" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001300 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19563c05-8911-48a9-bdf0-ca368119b49d" volumeName="kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-config" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001311 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fa778a3-926f-46e5-8a16-976b7f9819ca" volumeName="kubernetes.io/projected/4fa778a3-926f-46e5-8a16-976b7f9819ca-kube-api-access-nk8cd" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001322 5030 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea9c729-9323-4535-bb3b-f5e67bb28b29" volumeName="kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration" seLinuxMountContext="" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001333 5030 reconstruct.go:97] "Volume reconstruction finished" Oct 29 22:33:23.001303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001340 5030 reconciler.go:29] "Reconciler: start to sync state" Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.001993 5030 factory.go:153] Registering CRI-O factory Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002010 5030 factory.go:223] Registration of the crio container factory successfully Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002067 5030 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002078 5030 factory.go:55] Registering systemd factory Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002090 5030 factory.go:223] Registration of the systemd container factory successfully Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002112 5030 factory.go:103] Registering Raw factory Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002129 5030 manager.go:1199] Started watching for new ooms in manager Oct 29 22:33:23.007683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.002504 5030 manager.go:319] Starting recovery of all containers Oct 29 22:33:23.022578 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.022556 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/1.log" Oct 29 22:33:23.023946 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.022913 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/1.log" Oct 29 22:33:23.035163 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.035139 5030 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 29 22:33:23.036931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.036911 5030 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 29 22:33:23.036931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.036932 5030 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 29 22:33:23.037062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.036952 5030 kubelet.go:2521] "Starting kubelet main sync loop" Oct 29 22:33:23.037062 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:23.036990 5030 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 29 22:33:23.048984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.048955 5030 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="82be52e091ce1b3dc712d38781d22b4d78538ff806e6d548b0aa6ae4d6a0cc4e" exitCode=0 Oct 29 22:33:23.049056 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.048985 5030 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="8d3c9dc4f280f416caf642a7cba3c0a35a80ffdf98ad8c0639bee720b748c064" exitCode=0 Oct 29 22:33:23.049056 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.048996 5030 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="df2a14be32fa37a4210d9d51000ddbbc9268d98d9a671178bd22642500079fdd" exitCode=0 Oct 29 22:33:23.049056 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.049006 5030 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="e0f0ce8ddb2c67bd27ba912d1ac89307e96a9779625905e823cbfa002fb52a79" exitCode=0 Oct 29 22:33:23.049056 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.049015 5030 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="d5ab3323b943d6d0cf2d930ee3af615a8724deec55259f6b47cfe7949cc5fc3d" exitCode=0 Oct 29 22:33:23.049056 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.049023 5030 generic.go:356] "Generic (PLEG): container finished" podID="b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8" containerID="4889c30a96688726f8d0559eaeaa09d0067ffabaa2d4f8d2804094288655d8fc" exitCode=0 Oct 29 22:33:23.051300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.051278 5030 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:33:23.053641 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.053621 5030 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 29 22:33:23.053814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.053794 5030 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:33:23.068329 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.068311 5030 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:33:23.068718 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.068684 5030 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 22:33:23.076407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.076388 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/1.log" Oct 29 22:33:23.076504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.076430 5030 generic.go:356] "Generic (PLEG): container finished" podID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" exitCode=255 Oct 29 22:33:23.086333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.086266 5030 generic.go:356] "Generic (PLEG): container finished" podID="ff7d2aee7d03f12892682e2d7e593f68" containerID="467428ddcd2b4b2f0450654c0b028a9438b3aab95a713986690471e1898c512d" exitCode=0 Oct 29 22:33:23.110939 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.110922 5030 manager.go:324] Recovery completed Oct 29 22:33:23.131910 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.131888 5030 cpu_manager.go:226] "Starting" policy="none" Oct 29 22:33:23.131910 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.131906 5030 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 29 22:33:23.132025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.131929 5030 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 29 22:33:23.132086 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132073 5030 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 29 22:33:23.132138 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132088 5030 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 29 22:33:23.132138 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132118 5030 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 29 22:33:23.132138 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132127 5030 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 29 22:33:23.132138 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132134 5030 policy_none.go:50] "Start" Oct 29 22:33:23.132243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132142 5030 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 29 22:33:23.132243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132152 5030 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 29 22:33:23.132300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132278 5030 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 29 22:33:23.132300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132284 5030 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 29 22:33:23.132300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.132290 5030 policy_none.go:44] "Start" Oct 29 22:33:23.136568 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.136542 5030 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 29 22:33:23.137070 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:23.137044 5030 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 29 22:33:23.139112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.139097 5030 manager.go:341] "Starting Device Plugin manager" Oct 29 22:33:23.139566 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:23.139548 5030 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 29 22:33:23.139566 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.139566 5030 server.go:87] "Starting device plugin registration server" Oct 29 22:33:23.139861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.139831 5030 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 29 22:33:23.139927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.139862 5030 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 29 22:33:23.139975 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.139950 5030 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 29 22:33:23.140120 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.140083 5030 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 29 22:33:23.140198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.140129 5030 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 29 22:33:23.140198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.140137 5030 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 29 22:33:23.140296 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.140190 5030 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T22:33:23.140117771Z","UUID":"9d82617b-f679-484a-94fd-b54d1d27d09a","Handler":null,"Name":"","Endpoint":""} Oct 29 22:33:23.141655 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:23.141635 5030 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 29 22:33:23.142559 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.142541 5030 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 29 22:33:23.142660 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.142578 5030 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 29 22:33:23.240429 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.240405 5030 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 22:33:23.241615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.241596 5030 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientMemory" Oct 29 22:33:23.241716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.241633 5030 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 22:33:23.241716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.241647 5030 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-228.ec2.internal" event="NodeHasSufficientPID" Oct 29 22:33:23.241716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.241685 5030 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.279066 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.279048 5030 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.279150 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.279105 5030 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.337873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.337788 5030 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal"] Oct 29 22:33:23.338434 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338421 5030 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.338488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338411 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" event={"ID":"2e2359a413d1cf6872204355c65cbec7","Type":"ContainerStarted","Data":"2775dd6826ec442e90fe707cde3ce6a7b82235071831f5d4d476d8132944f15d"} Oct 29 22:33:23.338488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338453 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" event={"ID":"2e2359a413d1cf6872204355c65cbec7","Type":"ContainerStarted","Data":"23f6bcee7b0ab50a1a9c3694555bddc4ebe8a1508a05f145ab4918de2d7bf4f2"} Oct 29 22:33:23.338488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338473 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" event={"ID":"ff7d2aee7d03f12892682e2d7e593f68","Type":"ContainerStarted","Data":"1fa86ab569d8ee8b64d8cc429df5b4e001d034b86c6f14149fd5a5a2c5170a4f"} Oct 29 22:33:23.338584 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338493 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" event={"ID":"ff7d2aee7d03f12892682e2d7e593f68","Type":"ContainerDied","Data":"467428ddcd2b4b2f0450654c0b028a9438b3aab95a713986690471e1898c512d"} Oct 29 22:33:23.338584 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338510 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" event={"ID":"ff7d2aee7d03f12892682e2d7e593f68","Type":"ContainerStarted","Data":"75f1c04d20d6f1ed0f647265a2c27029f333e2836a8804c3c200a634093469c7"} Oct 29 22:33:23.338649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.338629 5030 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.365475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.365456 5030 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 22:33:23.365557 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:23.365490 5030 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.366082 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.366069 5030 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 22:33:23.366120 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:23.366100 5030 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.455573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.455516 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.455650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.455580 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.455650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.455627 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e2359a413d1cf6872204355c65cbec7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" (UID: \"2e2359a413d1cf6872204355c65cbec7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.556275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.556247 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.556363 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.556282 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.556363 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.556307 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e2359a413d1cf6872204355c65cbec7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" (UID: \"2e2359a413d1cf6872204355c65cbec7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.556496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.556360 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.556496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.556365 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff7d2aee7d03f12892682e2d7e593f68-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" (UID: \"ff7d2aee7d03f12892682e2d7e593f68\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.556496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.556425 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2e2359a413d1cf6872204355c65cbec7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" (UID: \"2e2359a413d1cf6872204355c65cbec7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:23.982436 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:23.982409 5030 apiserver.go:52] "Watching apiserver" Oct 29 22:33:24.043518 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.043485 5030 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 29 22:33:24.044211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.044182 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb","openshift-image-registry/image-registry-7fdff46b97-drfgt","openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp","openshift-network-diagnostics/network-check-target-rh76g","openshift-network-operator/iptables-alerter-wc75t","openshift-insights/periodic-gathering-s72v6-5hwwc","kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal","openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d","openshift-multus/multus-hf4d4","openshift-insights/insights-runtime-extractor-tdvz7","openshift-ingress/router-default-7b6bb75897-sr2f4","openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p","openshift-service-ca/service-ca-5fffc97bdf-6564h","openshift-image-registry/image-registry-66bf65fd8-phdn2","openshift-image-registry/node-ca-gkr9j","kube-system/konnectivity-agent-skdsx","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d","openshift-insights/insights-operator-7c4986d449-8p2vc","openshift-console-operator/console-operator-7cd56b8dbc-m855q","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk","kube-system/global-pull-secret-syncer-65b9p","kube-system/kas-connection-checker-685895c7c9-6z7h7","openshift-cluster-node-tuning-operator/tuned-wh7pt","openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh","openshift-dns/node-resolver-9655m","openshift-dns/dns-default-vml7h","openshift-ingress-canary/ingress-canary-rrr8p","openshift-network-console/networking-console-plugin-86c478bb88-w564k","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x","openshift-multus/multus-additional-cni-plugins-wcwmt","kube-system/kas-connection-checker-685895c7c9-xtkwb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-prpm2","openshift-multus/network-metrics-daemon-qtg9m","kube-system/kas-connection-checker-685895c7c9-jlpb2"] Oct 29 22:33:24.045813 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.045792 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.046155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.046138 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.046517 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.046480 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.047272 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.047255 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.051360 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.051326 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.052924 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.052909 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:24.057591 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.057578 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.058057 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.058043 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.060814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.060796 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.061828 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.061808 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 29 22:33:24.062064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062047 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.062263 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062247 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 29 22:33:24.062343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062276 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.062343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062282 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 29 22:33:24.062471 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062458 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.062676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062658 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 29 22:33:24.062707 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062662 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 29 22:33:24.062831 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062819 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 29 22:33:24.062959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.062944 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 29 22:33:24.063142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.063128 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.064140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.064126 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.064450 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.064437 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.064526 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.064514 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.065209 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.065195 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 29 22:33:24.065265 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.065219 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.065348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.065299 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 29 22:33:24.065348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.065304 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.065546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.065533 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 29 22:33:24.072945 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.072926 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.075066 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.075053 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.075240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.075209 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 29 22:33:24.075599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.075578 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.075679 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.075627 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.077519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.077498 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 29 22:33:24.077634 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.077573 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-528cc\"" Oct 29 22:33:24.077698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.077658 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.078212 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078190 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ggjqq\"" Oct 29 22:33:24.078294 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078251 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 29 22:33:24.078294 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078264 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.078423 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078287 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xx9bd\"" Oct 29 22:33:24.078584 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078567 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.078639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078608 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 29 22:33:24.078684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078644 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j7jzk\"" Oct 29 22:33:24.078684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078654 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 29 22:33:24.078796 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078766 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-dpwt7\"" Oct 29 22:33:24.078870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078824 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 29 22:33:24.078948 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.078933 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 29 22:33:24.079061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079044 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.079268 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079252 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.079360 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079047 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j2wnd\"" Oct 29 22:33:24.079561 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079546 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 29 22:33:24.079639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079563 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.079639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079584 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 29 22:33:24.079639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079633 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-6bbfq\"" Oct 29 22:33:24.079785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079758 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 29 22:33:24.079871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079808 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 29 22:33:24.079926 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079882 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.079926 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079920 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.080031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079939 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 29 22:33:24.080031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079993 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 29 22:33:24.080134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080087 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 29 22:33:24.080134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080114 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 29 22:33:24.080316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080296 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-56msd\"" Oct 29 22:33:24.080316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080316 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-4f7f9\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080352 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bd868\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080362 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079758 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079885 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080429 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080438 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jmngh\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080455 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.079994 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080467 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080509 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080517 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pdbx4\"" Oct 29 22:33:24.080519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080525 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 29 22:33:24.081085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080532 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qj89s\"" Oct 29 22:33:24.081085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080572 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 29 22:33:24.081085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080577 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.081085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080603 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.081085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080620 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.081085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.080635 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j5zfb\"" Oct 29 22:33:24.081306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081294 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 29 22:33:24.081368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081302 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5fdr\"" Oct 29 22:33:24.081368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081304 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gqfpm\"" Oct 29 22:33:24.081368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081354 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9vstz\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081399 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-7wdgt\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081442 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081456 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081460 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8pxqn\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081473 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081498 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-shpd4\"" Oct 29 22:33:24.081564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081557 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081572 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081608 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081633 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081647 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p7zhx\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081636 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081670 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081676 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081695 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081634 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081714 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081728 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081730 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081739 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081758 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081762 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081770 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081789 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081807 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081821 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081884 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lsqkb\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081905 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rhrpc\"" Oct 29 22:33:24.082006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.081946 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 29 22:33:24.082704 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.082286 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 29 22:33:24.091640 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.091623 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 29 22:33:24.091728 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.091715 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 29 22:33:24.099362 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.099346 5030 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 29 22:33:24.101704 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.101690 5030 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:24.101869 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.101854 5030 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:24.123936 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.123914 5030 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 22:33:24.124019 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.123961 5030 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-228.ec2.internal" Oct 29 22:33:24.124019 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.123914 5030 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 22:33:24.124114 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.124036 5030 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-228.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-228.ec2.internal" Oct 29 22:33:24.137635 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.137606 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:24.159176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159155 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87h4\" (UniqueName: \"kubernetes.io/projected/d8b45469-5973-49bc-b8c2-8ec0dce0e242-kube-api-access-q87h4\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:24.159243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159182 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3374d1d7-08e1-409a-8361-cf995598345b-crio-socket\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.159243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159201 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.159243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159216 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-multus\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.159379 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159318 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-kubelet\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.159447 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159418 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.159494 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159479 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-kubelet-config\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.159529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159516 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-system-cni-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.159558 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159539 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-serving-cert\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.159590 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159562 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.159590 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159579 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-trusted-ca\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.159644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159609 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.159673 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159644 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.159715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159703 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-serving-cert\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.159715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159704 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.159776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159732 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnxbw\" (UniqueName: \"kubernetes.io/projected/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-kube-api-access-gnxbw\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.159776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159762 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-key\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:24.159879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159784 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzfch\" (UniqueName: \"kubernetes.io/projected/19563c05-8911-48a9-bdf0-ca368119b49d-kube-api-access-zzfch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.159879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159806 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvg8\" (UniqueName: \"kubernetes.io/projected/64f11246-8026-4e31-b461-1768145983a5-kube-api-access-hwvg8\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.159879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159826 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa7fc2d-d350-4aba-977b-32868a6e3fe7-tmp-dir\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159908 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-netns\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159924 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa7fc2d-d350-4aba-977b-32868a6e3fe7-tmp-dir\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159939 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159943 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-script-lib\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159972 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-binary-copy\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159984 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwxrx\" (UniqueName: \"kubernetes.io/projected/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9-kube-api-access-zwxrx\") pod \"volume-data-source-validator-58c9f5f46d-829vk\" (UID: \"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:24.160052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.159993 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-trusted-ca\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160054 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-tmp\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160102 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160115 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-tmp\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160135 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160166 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-ovn\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160174 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-key\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160190 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-config\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160230 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160242 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160260 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160272 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-script-lib\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160289 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-systemd\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160312 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-host\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.160359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160354 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-node-log\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160370 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-ovnkube-config\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160388 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bba55700-7326-482d-919f-02f3eff316a1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160389 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160424 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160480 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f37d2fc-3cf7-4105-9c73-bf532a71af91-original-pull-secret\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160495 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvcvp\" (UniqueName: \"kubernetes.io/projected/8978b075-27a6-4c34-b4d4-36270f768a01-kube-api-access-nvcvp\") pod \"migrator-66df5d496b-kjspb\" (UID: \"8978b075-27a6-4c34-b4d4-36270f768a01\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160560 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bba55700-7326-482d-919f-02f3eff316a1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160568 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpmqc\" (UniqueName: \"kubernetes.io/projected/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20-kube-api-access-zpmqc\") pod \"kas-connection-checker-685895c7c9-xtkwb\" (UID: \"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20\") " pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160627 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bzv6\" (UniqueName: \"kubernetes.io/projected/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-kube-api-access-6bzv6\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160657 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-konnectivity-ca\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160687 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160736 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160767 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw9pj\" (UniqueName: \"kubernetes.io/projected/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-kube-api-access-vw9pj\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160784 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160804 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160880 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqblx\" (UniqueName: \"kubernetes.io/projected/8db65a63-2853-43db-8877-83a0d26c2de2-kube-api-access-wqblx\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160916 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-kubernetes\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.160950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160941 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-host\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160977 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-tuned\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160997 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.160974 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-konnectivity-ca\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161030 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-systemd-units\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161075 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-tuned\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161084 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-slash\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161075 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161115 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac5c5e2-a666-4cbf-9a73-677929dc640e-tmp-dir\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161135 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t79s\" (UniqueName: \"kubernetes.io/projected/efa7fc2d-d350-4aba-977b-32868a6e3fe7-kube-api-access-8t79s\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161160 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac5c5e2-a666-4cbf-9a73-677929dc640e-tmp-dir\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161161 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161190 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161215 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-hostroot\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161253 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-etc-kubernetes\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161293 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3374d1d7-08e1-409a-8361-cf995598345b-data-volume\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161358 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161366 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysconfig\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161403 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161362 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3374d1d7-08e1-409a-8361-cf995598345b-data-volume\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161440 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-config\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161465 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-socket-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161496 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-systemd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161526 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkht6\" (UniqueName: \"kubernetes.io/projected/5332e2aa-b433-47d6-b3c3-d42e66376eca-kube-api-access-mkht6\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161551 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161571 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161592 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-multus-certs\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161600 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-config\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161611 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-var-lib-kubelet\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161636 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrrj\" (UniqueName: \"kubernetes.io/projected/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-kube-api-access-7mrrj\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161651 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmq6n\" (UniqueName: \"kubernetes.io/projected/1ac5c5e2-a666-4cbf-9a73-677929dc640e-kube-api-access-nmq6n\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161692 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-sys\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161722 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-lib-modules\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161766 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbrmd\" (UniqueName: \"kubernetes.io/projected/901d2e7b-558b-4fc4-833b-a71699d02494-kube-api-access-bbrmd\") pod \"migrator-66df5d496b-n6zwh\" (UID: \"901d2e7b-558b-4fc4-833b-a71699d02494\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161801 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161830 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-k8s-cni-cncf-io\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161892 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hkth\" (UniqueName: \"kubernetes.io/projected/24316779-cb78-42df-954b-c36d2b0e097d-kube-api-access-8hkth\") pod \"network-check-source-6f78d59f99-mms8d\" (UID: \"24316779-cb78-42df-954b-c36d2b0e097d\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161919 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-serviceca\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.161931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161945 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161970 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.161996 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162026 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-cnibin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162083 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162111 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8cd\" (UniqueName: \"kubernetes.io/projected/4fa778a3-926f-46e5-8a16-976b7f9819ca-kube-api-access-nk8cd\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162141 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3a898c-87c6-409d-a70f-f01e8e15d787-serving-cert\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162149 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-serviceca\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162168 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162183 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162262 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-sys-fs\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162280 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162317 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-var-lib-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162349 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-log-socket\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162383 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162450 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-registration-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162477 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-dbus\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162498 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-netns\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162504 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162524 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3a898c-87c6-409d-a70f-f01e8e15d787-serving-cert\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162536 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ec29132-5b42-4a06-be85-de3789f00145-serving-cert\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162570 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-tmp\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162589 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khhs6\" (UniqueName: \"kubernetes.io/projected/ed3a898c-87c6-409d-a70f-f01e8e15d787-kube-api-access-khhs6\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162608 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f11246-8026-4e31-b461-1768145983a5-host-slash\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162629 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsb72\" (UniqueName: \"kubernetes.io/projected/3374d1d7-08e1-409a-8361-cf995598345b-kube-api-access-zsb72\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162749 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162762 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-tmp\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162783 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162794 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ec29132-5b42-4a06-be85-de3789f00145-serving-cert\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162811 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162853 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-cni-binary-copy\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162882 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-bin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162885 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162935 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-conf\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162950 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162962 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162962 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.162993 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-bin\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163011 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163048 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-cabundle\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163085 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rx5d\" (UniqueName: \"kubernetes.io/projected/24348922-a2e2-4e39-9166-1aed0076055c-kube-api-access-7rx5d\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163098 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-cni-binary-copy\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163113 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-netd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163139 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-service-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163165 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8b45469-5973-49bc-b8c2-8ec0dce0e242-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163189 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ec29132-5b42-4a06-be85-de3789f00145-config\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163203 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-signing-cabundle\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163209 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-agent-certs\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163229 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cnibin\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163244 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-socket-dir-parent\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163262 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-conf-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163283 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-run\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163288 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3a898c-87c6-409d-a70f-f01e8e15d787-service-ca-bundle\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163326 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b45469-5973-49bc-b8c2-8ec0dce0e242-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163330 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4ec29132-5b42-4a06-be85-de3789f00145-config\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163344 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8b45469-5973-49bc-b8c2-8ec0dce0e242-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163371 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163436 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8b45469-5973-49bc-b8c2-8ec0dce0e242-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163441 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw8wf\" (UniqueName: \"kubernetes.io/projected/4ec29132-5b42-4a06-be85-de3789f00145-kube-api-access-lw8wf\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:24.163476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163479 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5092efe-96fd-4aec-99f1-a2c07c84ac7f-agent-certs\") pod \"konnectivity-agent-skdsx\" (UID: \"e5092efe-96fd-4aec-99f1-a2c07c84ac7f\") " pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163516 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-etc-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163546 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163566 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163591 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163618 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ghgf\" (UniqueName: \"kubernetes.io/projected/230e7bac-638f-4478-989a-abad8f7213bd-kube-api-access-4ghgf\") pod \"kas-connection-checker-685895c7c9-6z7h7\" (UID: \"230e7bac-638f-4478-989a-abad8f7213bd\") " pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163665 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-kubelet\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163783 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/64f11246-8026-4e31-b461-1768145983a5-iptables-alerter-script\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163805 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163832 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-device-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163872 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-env-overrides\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163895 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.163924 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164030 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164032 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164040 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/64f11246-8026-4e31-b461-1768145983a5-iptables-alerter-script\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164060 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19563c05-8911-48a9-bdf0-ca368119b49d-ovn-node-metrics-cert\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164067 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19563c05-8911-48a9-bdf0-ca368119b49d-env-overrides\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164077 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjjq\" (UniqueName: \"kubernetes.io/projected/983a2044-473f-48be-8e73-5da2ed82aaf4-kube-api-access-bpjjq\") pod \"kas-connection-checker-685895c7c9-jlpb2\" (UID: \"983a2044-473f-48be-8e73-5da2ed82aaf4\") " pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164111 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164113 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164145 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8db65a63-2853-43db-8877-83a0d26c2de2-cert\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164161 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19563c05-8911-48a9-bdf0-ca368119b49d-ovn-node-metrics-cert\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164240 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164266 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-daemon-config\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164281 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-snapshots\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164293 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8db65a63-2853-43db-8877-83a0d26c2de2-cert\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164302 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164352 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed3a898c-87c6-409d-a70f-f01e8e15d787-snapshots\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164403 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac5c5e2-a666-4cbf-9a73-677929dc640e-hosts-file\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164423 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164435 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-daemon-config\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164455 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164504 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-modprobe-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164521 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxs2w\" (UniqueName: \"kubernetes.io/projected/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-kube-api-access-cxs2w\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164537 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb729\" (UniqueName: \"kubernetes.io/projected/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-kube-api-access-vb729\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164561 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8wxt\" (UniqueName: \"kubernetes.io/projected/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-kube-api-access-g8wxt\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164570 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164659 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-default-certificate\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164717 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164753 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-stats-auth\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164798 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efa7fc2d-d350-4aba-977b-32868a6e3fe7-config-volume\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164830 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-system-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164901 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164919 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-default-certificate\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164945 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-stats-auth\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164960 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164988 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-557gg\" (UniqueName: \"kubernetes.io/projected/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-kube-api-access-557gg\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.165556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.164993 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efa7fc2d-d350-4aba-977b-32868a6e3fe7-config-volume\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.180672 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.180649 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnxbw\" (UniqueName: \"kubernetes.io/projected/5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb-kube-api-access-gnxbw\") pod \"console-operator-7cd56b8dbc-m855q\" (UID: \"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:24.181037 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.181021 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87h4\" (UniqueName: \"kubernetes.io/projected/d8b45469-5973-49bc-b8c2-8ec0dce0e242-kube-api-access-q87h4\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d\" (UID: \"d8b45469-5973-49bc-b8c2-8ec0dce0e242\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" Oct 29 22:33:24.181152 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.181132 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvg8\" (UniqueName: \"kubernetes.io/projected/64f11246-8026-4e31-b461-1768145983a5-kube-api-access-hwvg8\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.181301 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.181287 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzfch\" (UniqueName: \"kubernetes.io/projected/19563c05-8911-48a9-bdf0-ca368119b49d-kube-api-access-zzfch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.184629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.184613 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") pod \"periodic-gathering-s72v6-5hwwc\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:33:24.185723 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.185707 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bzv6\" (UniqueName: \"kubernetes.io/projected/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-kube-api-access-6bzv6\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.186448 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.186433 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvcvp\" (UniqueName: \"kubernetes.io/projected/8978b075-27a6-4c34-b4d4-36270f768a01-kube-api-access-nvcvp\") pod \"migrator-66df5d496b-kjspb\" (UID: \"8978b075-27a6-4c34-b4d4-36270f768a01\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-kjspb" Oct 29 22:33:24.186652 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.186633 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw9pj\" (UniqueName: \"kubernetes.io/projected/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-kube-api-access-vw9pj\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:24.186740 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.186724 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqblx\" (UniqueName: \"kubernetes.io/projected/8db65a63-2853-43db-8877-83a0d26c2de2-kube-api-access-wqblx\") pod \"ingress-canary-rrr8p\" (UID: \"8db65a63-2853-43db-8877-83a0d26c2de2\") " pod="openshift-ingress-canary/ingress-canary-rrr8p" Oct 29 22:33:24.186826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.186813 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.187086 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.187071 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwxrx\" (UniqueName: \"kubernetes.io/projected/86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9-kube-api-access-zwxrx\") pod \"volume-data-source-validator-58c9f5f46d-829vk\" (UID: \"86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-829vk" Oct 29 22:33:24.188258 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.188238 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-557gg\" (UniqueName: \"kubernetes.io/projected/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-kube-api-access-557gg\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.189150 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.189129 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.189710 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t79s\" (UniqueName: \"kubernetes.io/projected/efa7fc2d-d350-4aba-977b-32868a6e3fe7-kube-api-access-8t79s\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190238 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjjq\" (UniqueName: \"kubernetes.io/projected/983a2044-473f-48be-8e73-5da2ed82aaf4-kube-api-access-bpjjq\") pod \"kas-connection-checker-685895c7c9-jlpb2\" (UID: \"983a2044-473f-48be-8e73-5da2ed82aaf4\") " pod="kube-system/kas-connection-checker-685895c7c9-jlpb2" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190320 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8wxt\" (UniqueName: \"kubernetes.io/projected/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-kube-api-access-g8wxt\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190545 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khhs6\" (UniqueName: \"kubernetes.io/projected/ed3a898c-87c6-409d-a70f-f01e8e15d787-kube-api-access-khhs6\") pod \"insights-operator-7c4986d449-8p2vc\" (UID: \"ed3a898c-87c6-409d-a70f-f01e8e15d787\") " pod="openshift-insights/insights-operator-7c4986d449-8p2vc" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190623 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw8wf\" (UniqueName: \"kubernetes.io/projected/4ec29132-5b42-4a06-be85-de3789f00145-kube-api-access-lw8wf\") pod \"service-ca-operator-544d749c8c-7jt8p\" (UID: \"4ec29132-5b42-4a06-be85-de3789f00145\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190823 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb729\" (UniqueName: \"kubernetes.io/projected/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-kube-api-access-vb729\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190886 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbrmd\" (UniqueName: \"kubernetes.io/projected/901d2e7b-558b-4fc4-833b-a71699d02494-kube-api-access-bbrmd\") pod \"migrator-66df5d496b-n6zwh\" (UID: \"901d2e7b-558b-4fc4-833b-a71699d02494\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zwh" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190953 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8cd\" (UniqueName: \"kubernetes.io/projected/4fa778a3-926f-46e5-8a16-976b7f9819ca-kube-api-access-nk8cd\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.190961 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.191123 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsb72\" (UniqueName: \"kubernetes.io/projected/3374d1d7-08e1-409a-8361-cf995598345b-kube-api-access-zsb72\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.191237 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mrrj\" (UniqueName: \"kubernetes.io/projected/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-kube-api-access-7mrrj\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.191941 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxs2w\" (UniqueName: \"kubernetes.io/projected/fffc6a30-b103-428e-a73e-7d7d03a4e2ff-kube-api-access-cxs2w\") pod \"service-ca-5fffc97bdf-6564h\" (UID: \"fffc6a30-b103-428e-a73e-7d7d03a4e2ff\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6564h" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.191960 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hkth\" (UniqueName: \"kubernetes.io/projected/24316779-cb78-42df-954b-c36d2b0e097d-kube-api-access-8hkth\") pod \"network-check-source-6f78d59f99-mms8d\" (UID: \"24316779-cb78-42df-954b-c36d2b0e097d\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-mms8d" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.192175 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmq6n\" (UniqueName: \"kubernetes.io/projected/1ac5c5e2-a666-4cbf-9a73-677929dc640e-kube-api-access-nmq6n\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.192293 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ghgf\" (UniqueName: \"kubernetes.io/projected/230e7bac-638f-4478-989a-abad8f7213bd-kube-api-access-4ghgf\") pod \"kas-connection-checker-685895c7c9-6z7h7\" (UID: \"230e7bac-638f-4478-989a-abad8f7213bd\") " pod="kube-system/kas-connection-checker-685895c7c9-6z7h7" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.192528 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rx5d\" (UniqueName: \"kubernetes.io/projected/24348922-a2e2-4e39-9166-1aed0076055c-kube-api-access-7rx5d\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.192609 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpmqc\" (UniqueName: \"kubernetes.io/projected/4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20-kube-api-access-zpmqc\") pod \"kas-connection-checker-685895c7c9-xtkwb\" (UID: \"4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20\") " pod="kube-system/kas-connection-checker-685895c7c9-xtkwb" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.192664 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25p5w\" (UniqueName: \"kubernetes.io/projected/aa67ab99-04d4-4f83-a61b-d6f8fe0b0032-kube-api-access-25p5w\") pod \"network-check-target-rh76g\" (UID: \"aa67ab99-04d4-4f83-a61b-d6f8fe0b0032\") " pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:24.194218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.194142 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.195234 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.195213 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkht6\" (UniqueName: \"kubernetes.io/projected/5332e2aa-b433-47d6-b3c3-d42e66376eca-kube-api-access-mkht6\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.261806 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.261750 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:24.265615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265591 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.265673 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265625 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.265673 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265643 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac5c5e2-a666-4cbf-9a73-677929dc640e-hosts-file\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.265673 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265658 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265677 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-modprobe-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265701 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-system-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265714 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265746 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265764 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ac5c5e2-a666-4cbf-9a73-677929dc640e-hosts-file\") pod \"node-resolver-9655m\" (UID: \"1ac5c5e2-a666-4cbf-9a73-677929dc640e\") " pod="openshift-dns/node-resolver-9655m" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265781 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-system-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265772 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3374d1d7-08e1-409a-8361-cf995598345b-crio-socket\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.265820 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.265788 5030 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265827 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-multus\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265865 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-modprobe-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265880 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-kubelet\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265861 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3374d1d7-08e1-409a-8361-cf995598345b-crio-socket\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.265913 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:24.765892805 +0000 UTC m=+1.878538390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265917 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-d\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265928 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-kubelet\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265938 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-multus\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265947 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265972 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-kubelet-config\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.265987 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-system-cni-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266005 5030 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266047 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-system-cni-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266052 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-kubelet-config\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266008 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266099 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-netns\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.266119 5030 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266126 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-ovn\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266168 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-ovn\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266185 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-netns\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.266192 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:24.766157726 +0000 UTC m=+1.878803297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:24.266211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266226 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-systemd\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266251 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-host\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266274 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-node-log\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266277 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-systemd\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266295 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7881bd2-9f24-4c58-9a77-68cc091ee9ab-host\") pod \"node-ca-gkr9j\" (UID: \"c7881bd2-9f24-4c58-9a77-68cc091ee9ab\") " pod="openshift-image-registry/node-ca-gkr9j" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266304 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266330 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266342 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-node-log\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266354 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-kubernetes\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266375 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-host\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266406 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-run-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266423 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-kubernetes\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266425 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-host\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266442 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-systemd-units\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266467 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-slash\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266492 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266517 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-hostroot\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266538 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-etc-kubernetes\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266565 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysconfig\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266590 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-socket-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266596 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-etc-kubernetes\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266612 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-systemd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266539 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-slash\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266567 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-systemd-units\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.266647 5030 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.266708 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:24.766692305 +0000 UTC m=+1.879337871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266807 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266852 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-multus-certs\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266859 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-run-systemd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266904 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-var-lib-kubelet\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266918 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266931 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-sys\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266954 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-lib-modules\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266975 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysconfig\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266980 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.266996 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-var-lib-kubelet\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267018 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-k8s-cni-cncf-io\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.267045 5030 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267012 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-sys\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.267098 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:24.767082623 +0000 UTC m=+1.879728192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : secret "metrics-daemon-secret" not found Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267105 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-lib-modules\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267140 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-k8s-cni-cncf-io\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267155 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267069 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267181 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-multus-certs\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267202 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-cnibin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.267202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267215 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-hostroot\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267247 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-socket-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267263 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267270 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-cnibin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267292 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-sys-fs\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267384 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-var-lib-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267432 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-log-socket\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267448 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-var-lib-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267470 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-sys-fs\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267523 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-log-socket\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267534 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-registration-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267564 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-dbus\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267587 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-registration-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267596 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-netns\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267623 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f11246-8026-4e31-b461-1768145983a5-host-slash\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267647 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f37d2fc-3cf7-4105-9c73-bf532a71af91-dbus\") pod \"global-pull-secret-syncer-65b9p\" (UID: \"3f37d2fc-3cf7-4105-9c73-bf532a71af91\") " pod="kube-system/global-pull-secret-syncer-65b9p" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267649 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-bin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267682 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-var-lib-cni-bin\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267686 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-conf\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267702 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-host-run-netns\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267715 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267743 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-bin\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267771 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-netd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267784 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267745 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64f11246-8026-4e31-b461-1768145983a5-host-slash\") pod \"iptables-alerter-wc75t\" (UID: \"64f11246-8026-4e31-b461-1768145983a5\") " pod="openshift-network-operator/iptables-alerter-wc75t" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267800 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-etc-sysctl-conf\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267800 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cnibin\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267860 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-bin\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267863 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8-cnibin\") pod \"multus-additional-cni-plugins-wcwmt\" (UID: \"b6ca05c6-f2bd-4ada-b0a1-b9cd07a89ff8\") " pod="openshift-multus/multus-additional-cni-plugins-wcwmt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267867 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-socket-dir-parent\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267900 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-conf-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267908 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-cni-netd\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267914 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-socket-dir-parent\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267925 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-run\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267950 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267954 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-conf-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.267976 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-etc-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268000 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268025 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268041 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-etc-openvswitch\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268049 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-kubelet\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268003 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c8a747-4790-4ff6-9c11-a1b66d5a98c3-run\") pod \"tuned-wh7pt\" (UID: \"28c8a747-4790-4ff6-9c11-a1b66d5a98c3\") " pod="openshift-cluster-node-tuning-operator/tuned-wh7pt" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268076 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-device-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268099 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268127 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268204 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268383 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19563c05-8911-48a9-bdf0-ca368119b49d-host-kubelet\") pod \"ovnkube-node-prpm2\" (UID: \"19563c05-8911-48a9-bdf0-ca368119b49d\") " pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268448 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/24348922-a2e2-4e39-9166-1aed0076055c-device-dir\") pod \"aws-ebs-csi-driver-node-pmv8x\" (UID: \"24348922-a2e2-4e39-9166-1aed0076055c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pmv8x" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268506 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fa778a3-926f-46e5-8a16-976b7f9819ca-multus-cni-dir\") pod \"multus-hf4d4\" (UID: \"4fa778a3-926f-46e5-8a16-976b7f9819ca\") " pod="openshift-multus/multus-hf4d4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268536 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3374d1d7-08e1-409a-8361-cf995598345b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268736 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-service-ca-bundle\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268876 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5332e2aa-b433-47d6-b3c3-d42e66376eca-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-gwr5t\" (UID: \"5332e2aa-b433-47d6-b3c3-d42e66376eca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.268895 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ce7d6cc-47fa-4a6a-ba52-5a23efd31019-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lmjqp\" (UID: \"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.269435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.269354 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bba55700-7326-482d-919f-02f3eff316a1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-w564k\" (UID: \"bba55700-7326-482d-919f-02f3eff316a1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.271412 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.270796 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"image-registry-66bf65fd8-phdn2\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.271412 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.270972 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"image-registry-7fdff46b97-drfgt\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.346465 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.346437 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" Oct 29 22:33:24.347064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.347049 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" Oct 29 22:33:24.348229 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.348214 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:24.348293 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.348282 5030 scope.go:122] "RemoveContainer" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" Oct 29 22:33:24.351931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.351916 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" Oct 29 22:33:24.357803 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.357781 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:24.366972 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.366950 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:24.582077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.582051 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:24.664258 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.664232 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp"] Oct 29 22:33:24.668215 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:24.668183 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce7d6cc_47fa_4a6a_ba52_5a23efd31019.slice/crio-464ddaaacb8789f632c80f3ed87441230ec51365e896edad6a52b4fe2576eb06 WatchSource:0}: Error finding container 464ddaaacb8789f632c80f3ed87441230ec51365e896edad6a52b4fe2576eb06: Status 404 returned error can't find the container with id 464ddaaacb8789f632c80f3ed87441230ec51365e896edad6a52b4fe2576eb06 Oct 29 22:33:24.670124 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.670102 5030 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 22:33:24.702299 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.702275 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t"] Oct 29 22:33:24.753142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.753118 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fdff46b97-drfgt"] Oct 29 22:33:24.755532 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:24.755508 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05dc76a9_2747_4bd0_907d_a9b2caa3b25d.slice/crio-86d2e23d329dab9463c2e46921ded8cf379bb7e5ef5c4e931ac4912d5e0044fb WatchSource:0}: Error finding container 86d2e23d329dab9463c2e46921ded8cf379bb7e5ef5c4e931ac4912d5e0044fb: Status 404 returned error can't find the container with id 86d2e23d329dab9463c2e46921ded8cf379bb7e5ef5c4e931ac4912d5e0044fb Oct 29 22:33:24.772614 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.772594 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:24.772696 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.772650 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:24.772696 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.772675 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:24.772806 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.772707 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:24.772806 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772760 5030 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 22:33:24.772806 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772789 5030 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:24.772960 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772813 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:25.772795069 +0000 UTC m=+2.885440651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : secret "metrics-daemon-secret" not found Oct 29 22:33:24.772960 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772814 5030 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:24.772960 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772830 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:25.772819568 +0000 UTC m=+2.885465135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:24.772960 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772869 5030 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:24.772960 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772898 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:25.772882464 +0000 UTC m=+2.885528046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:24.772960 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:24.772920 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:25.772910003 +0000 UTC m=+2.885555574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:24.784691 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.784667 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-w564k"] Oct 29 22:33:24.788149 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.788126 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66bf65fd8-phdn2"] Oct 29 22:33:24.788429 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:24.788409 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba55700_7326_482d_919f_02f3eff316a1.slice/crio-4e9fac5c4cce8b8d84e60877284f1df4cc22900455497017c3fbcacbeac66de7 WatchSource:0}: Error finding container 4e9fac5c4cce8b8d84e60877284f1df4cc22900455497017c3fbcacbeac66de7: Status 404 returned error can't find the container with id 4e9fac5c4cce8b8d84e60877284f1df4cc22900455497017c3fbcacbeac66de7 Oct 29 22:33:24.791175 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:24.791146 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea9c729_9323_4535_bb3b_f5e67bb28b29.slice/crio-8aa20cb35c8de461b04f7734a78cabd69f02f446d602fb10ae3477d330b00769 WatchSource:0}: Error finding container 8aa20cb35c8de461b04f7734a78cabd69f02f446d602fb10ae3477d330b00769: Status 404 returned error can't find the container with id 8aa20cb35c8de461b04f7734a78cabd69f02f446d602fb10ae3477d330b00769 Oct 29 22:33:24.865152 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.865127 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:24.982383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:24.982349 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:25.108698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.108612 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" event={"ID":"05dc76a9-2747-4bd0-907d-a9b2caa3b25d","Type":"ContainerStarted","Data":"2dcfa06cf84302aa68d234328a29b58428ac9250526dbcc8e5cc781e566ebe7f"} Oct 29 22:33:25.108698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.108671 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" event={"ID":"05dc76a9-2747-4bd0-907d-a9b2caa3b25d","Type":"ContainerStarted","Data":"86d2e23d329dab9463c2e46921ded8cf379bb7e5ef5c4e931ac4912d5e0044fb"} Oct 29 22:33:25.110280 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.110248 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" event={"ID":"bba55700-7326-482d-919f-02f3eff316a1","Type":"ContainerStarted","Data":"4e9fac5c4cce8b8d84e60877284f1df4cc22900455497017c3fbcacbeac66de7"} Oct 29 22:33:25.112576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.112552 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" event={"ID":"eea9c729-9323-4535-bb3b-f5e67bb28b29","Type":"ContainerStarted","Data":"595c3dcdf1ace8a094b81aa81d4dc749fdea78d369fa7e65ce62978d14234689"} Oct 29 22:33:25.112711 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.112607 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" event={"ID":"eea9c729-9323-4535-bb3b-f5e67bb28b29","Type":"ContainerStarted","Data":"8aa20cb35c8de461b04f7734a78cabd69f02f446d602fb10ae3477d330b00769"} Oct 29 22:33:25.114807 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.114777 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" event={"ID":"5332e2aa-b433-47d6-b3c3-d42e66376eca","Type":"ContainerStarted","Data":"c74a2240cd4e7d9a470fbfc30856b6195303d568a58275851cb8348664a9f34c"} Oct 29 22:33:25.116625 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.116603 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" event={"ID":"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019","Type":"ContainerStarted","Data":"464ddaaacb8789f632c80f3ed87441230ec51365e896edad6a52b4fe2576eb06"} Oct 29 22:33:25.118362 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.118342 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:33:25.118861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.118818 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/1.log" Oct 29 22:33:25.118955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.118874 5030 generic.go:356] "Generic (PLEG): container finished" podID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" containerID="80aeaa22340ccbbc72b8de4c6f7d07b9851aa16491010423826bf11813a06e03" exitCode=255 Oct 29 22:33:25.119119 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.119101 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" event={"ID":"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb","Type":"ContainerDied","Data":"80aeaa22340ccbbc72b8de4c6f7d07b9851aa16491010423826bf11813a06e03"} Oct 29 22:33:25.119186 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.119148 5030 scope.go:122] "RemoveContainer" containerID="42eede5433f0a8359c176a39860e395d9f1b22d3b570341ac366b38cc745788c" Oct 29 22:33:25.119539 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.119521 5030 scope.go:122] "RemoveContainer" containerID="80aeaa22340ccbbc72b8de4c6f7d07b9851aa16491010423826bf11813a06e03" Oct 29 22:33:25.119707 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.119687 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-m855q_openshift-console-operator(5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" Oct 29 22:33:25.168753 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.167697 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:25.767885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.767828 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:25.784069 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.784041 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:25.784230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.784100 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:25.784230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.784144 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:25.784230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.784210 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:25.784376 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784334 5030 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:25.784421 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784392 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.784375778 +0000 UTC m=+4.897021352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:25.784765 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784747 5030 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 22:33:25.784827 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784797 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs podName:e9fb3bd9-7312-4616-a8db-6e51b56d78a2 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.784783887 +0000 UTC m=+4.897429463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs") pod "router-default-7b6bb75897-sr2f4" (UID: "e9fb3bd9-7312-4616-a8db-6e51b56d78a2") : secret "router-metrics-certs-default" not found Oct 29 22:33:25.784932 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784865 5030 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:25.784932 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784894 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.78488422 +0000 UTC m=+4.897529789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:25.785029 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784941 5030 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 22:33:25.785029 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:25.784967 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs podName:4d6f9752-92e3-4e94-9ac0-a812fc12c7d8 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:27.784958196 +0000 UTC m=+4.897603766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs") pod "network-metrics-daemon-qtg9m" (UID: "4d6f9752-92e3-4e94-9ac0-a812fc12c7d8") : secret "metrics-daemon-secret" not found Oct 29 22:33:25.895459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:25.895423 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:26.124381 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.124339 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-w564k" event={"ID":"bba55700-7326-482d-919f-02f3eff316a1","Type":"ContainerStarted","Data":"1564b6aaac2f46674c9451a30c73e1abfb9ed2fe7fd3a1cf7adf904496ba4cf9"} Oct 29 22:33:26.126568 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.126547 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:33:26.126823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.126799 5030 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 22:33:26.174401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.174370 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:26.568904 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.568877 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:26.669476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.669450 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:26.972237 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:26.972201 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.119651 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.119621 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.132731 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.132700 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" event={"ID":"5332e2aa-b433-47d6-b3c3-d42e66376eca","Type":"ContainerStarted","Data":"e2417fd411e3710cbbc6d2f41420e104f46c23d0146acf9fe792ad85d7582003"} Oct 29 22:33:27.133183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.132746 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-gwr5t" event={"ID":"5332e2aa-b433-47d6-b3c3-d42e66376eca","Type":"ContainerStarted","Data":"12b3223b24d32ae51188c170d0d059d58a4754ae7da5730f1ffa8218d681130a"} Oct 29 22:33:27.134414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.134389 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lmjqp" event={"ID":"2ce7d6cc-47fa-4a6a-ba52-5a23efd31019","Type":"ContainerStarted","Data":"1c62334e3087e2be27fed1e7b63dc540b2e8dfd7b9191c17af2556c7e844f525"} Oct 29 22:33:27.240984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.240890 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.371575 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.371537 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.494355 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.494323 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.586681 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.586652 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.792231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.792206 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.805191 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.805170 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:27.805256 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.805208 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:27.805256 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.805249 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:27.805334 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.805281 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:27.805334 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:27.805313 5030 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 22:33:27.805390 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:27.805368 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls podName:3374d1d7-08e1-409a-8361-cf995598345b nodeName:}" failed. No retries permitted until 2026-10-29 22:33:31.805352135 +0000 UTC m=+8.917997706 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tdvz7" (UID: "3374d1d7-08e1-409a-8361-cf995598345b") : secret "insights-runtime-extractor-tls" not found Oct 29 22:33:27.805447 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:27.805431 5030 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 22:33:27.805510 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:27.805497 5030 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls podName:efa7fc2d-d350-4aba-977b-32868a6e3fe7 nodeName:}" failed. No retries permitted until 2026-10-29 22:33:31.805479651 +0000 UTC m=+8.918125217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls") pod "dns-default-vml7h" (UID: "efa7fc2d-d350-4aba-977b-32868a6e3fe7") : secret "dns-default-metrics-tls" not found Oct 29 22:33:27.807482 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.807462 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6f9752-92e3-4e94-9ac0-a812fc12c7d8-metrics-certs\") pod \"network-metrics-daemon-qtg9m\" (UID: \"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8\") " pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:27.807482 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.807479 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9fb3bd9-7312-4616-a8db-6e51b56d78a2-metrics-certs\") pod \"router-default-7b6bb75897-sr2f4\" (UID: \"e9fb3bd9-7312-4616-a8db-6e51b56d78a2\") " pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:27.904557 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.904523 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:27.947216 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.947192 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:27.954118 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:27.954098 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qtg9m" Oct 29 22:33:28.143817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.143782 5030 generic.go:356] "Generic (PLEG): container finished" podID="4ec29132-5b42-4a06-be85-de3789f00145" containerID="475c87c0fad0bba0eed642494df7438b7bbce2c4c92289dd8927b2b953089096" exitCode=0 Oct 29 22:33:28.144258 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.143866 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" event={"ID":"4ec29132-5b42-4a06-be85-de3789f00145","Type":"ContainerDied","Data":"475c87c0fad0bba0eed642494df7438b7bbce2c4c92289dd8927b2b953089096"} Oct 29 22:33:28.144258 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.144249 5030 scope.go:122] "RemoveContainer" containerID="475c87c0fad0bba0eed642494df7438b7bbce2c4c92289dd8927b2b953089096" Oct 29 22:33:28.145533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.145506 5030 generic.go:356] "Generic (PLEG): container finished" podID="ed3a898c-87c6-409d-a70f-f01e8e15d787" containerID="f9d3e2835828fd3d57f02bdc7028af74cff9a6e1b251467c3d0e3f75c0f364d7" exitCode=0 Oct 29 22:33:28.145635 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.145572 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" event={"ID":"ed3a898c-87c6-409d-a70f-f01e8e15d787","Type":"ContainerDied","Data":"f9d3e2835828fd3d57f02bdc7028af74cff9a6e1b251467c3d0e3f75c0f364d7"} Oct 29 22:33:28.146086 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.146070 5030 scope.go:122] "RemoveContainer" containerID="f9d3e2835828fd3d57f02bdc7028af74cff9a6e1b251467c3d0e3f75c0f364d7" Oct 29 22:33:28.181266 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.181240 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:28.267301 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.267271 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qtg9m"] Oct 29 22:33:28.270448 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:28.270421 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6f9752_92e3_4e94_9ac0_a812fc12c7d8.slice/crio-10a3e55a91edb4d2485a5606a52cb7c2d5edcad2c941094accac464df231ae8b WatchSource:0}: Error finding container 10a3e55a91edb4d2485a5606a52cb7c2d5edcad2c941094accac464df231ae8b: Status 404 returned error can't find the container with id 10a3e55a91edb4d2485a5606a52cb7c2d5edcad2c941094accac464df231ae8b Oct 29 22:33:28.270552 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.270537 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b6bb75897-sr2f4"] Oct 29 22:33:28.273893 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:28.273875 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fb3bd9_7312_4616_a8db_6e51b56d78a2.slice/crio-cb71cc3fe5c4a708c77a2d94193ce8c788bf6f3c80ca0ef27fc9af4f3bc56bd0 WatchSource:0}: Error finding container cb71cc3fe5c4a708c77a2d94193ce8c788bf6f3c80ca0ef27fc9af4f3bc56bd0: Status 404 returned error can't find the container with id cb71cc3fe5c4a708c77a2d94193ce8c788bf6f3c80ca0ef27fc9af4f3bc56bd0 Oct 29 22:33:28.569169 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.569083 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:28.789311 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.789279 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:28.789729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.789712 5030 scope.go:122] "RemoveContainer" containerID="80aeaa22340ccbbc72b8de4c6f7d07b9851aa16491010423826bf11813a06e03" Oct 29 22:33:28.789995 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:28.789968 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-m855q_openshift-console-operator(5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" Oct 29 22:33:28.976455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:28.976411 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:29.154018 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.153389 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" event={"ID":"ed3a898c-87c6-409d-a70f-f01e8e15d787","Type":"ContainerStarted","Data":"f78bf227f7d6997b403dba59695a13e0f442e113b14c9908a89bd45fd6b617bf"} Oct 29 22:33:29.155199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.155134 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qtg9m" event={"ID":"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8","Type":"ContainerStarted","Data":"10a3e55a91edb4d2485a5606a52cb7c2d5edcad2c941094accac464df231ae8b"} Oct 29 22:33:29.157038 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.157012 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" event={"ID":"e9fb3bd9-7312-4616-a8db-6e51b56d78a2","Type":"ContainerStarted","Data":"45d64093a242ff6b23c0f1b54aee852487afddbd3cb671cc3e44603d6fe0c437"} Oct 29 22:33:29.157167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.157058 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" event={"ID":"e9fb3bd9-7312-4616-a8db-6e51b56d78a2","Type":"ContainerStarted","Data":"cb71cc3fe5c4a708c77a2d94193ce8c788bf6f3c80ca0ef27fc9af4f3bc56bd0"} Oct 29 22:33:29.159158 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.159133 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-7jt8p" event={"ID":"4ec29132-5b42-4a06-be85-de3789f00145","Type":"ContainerStarted","Data":"7af1af1233b674e0b13578f5f09f6ef9a6ed01e25ea366031752b0763435c79b"} Oct 29 22:33:29.377048 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.377017 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:29.775035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.775007 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:29.938357 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.938326 5030 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:29.938687 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.938674 5030 scope.go:122] "RemoveContainer" containerID="80aeaa22340ccbbc72b8de4c6f7d07b9851aa16491010423826bf11813a06e03" Oct 29 22:33:29.938873 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:29.938832 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-m855q_openshift-console-operator(5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" Oct 29 22:33:29.947963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.947943 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:29.950383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:29.950367 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:30.117171 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.117107 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:30.117308 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.117296 5030 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 22:33:30.163386 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.163345 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:30.165065 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.165045 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qtg9m" event={"ID":"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8","Type":"ContainerStarted","Data":"507705706c057c6bc3755821e81edb5acb0070c0141946e09134dd4c938016e3"} Oct 29 22:33:30.165115 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.165093 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qtg9m" event={"ID":"4d6f9752-92e3-4e94-9ac0-a812fc12c7d8","Type":"ContainerStarted","Data":"c3b800e3b703ed64b5a7db1e6a81942ac9654bbafdea41df4d8978abd9400a9b"} Oct 29 22:33:30.569775 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.569740 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:30.992158 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:30.992127 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:31.127908 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.127879 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:31.161353 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.161326 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:31.168483 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.168462 5030 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 22:33:31.239607 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.239584 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:31.372125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.372069 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:31.548682 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.548660 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:31.811961 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.811940 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prpm2" Oct 29 22:33:31.842976 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.842944 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:31.843104 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.843006 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:31.845107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.845088 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efa7fc2d-d350-4aba-977b-32868a6e3fe7-metrics-tls\") pod \"dns-default-vml7h\" (UID: \"efa7fc2d-d350-4aba-977b-32868a6e3fe7\") " pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:31.845250 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.845225 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3374d1d7-08e1-409a-8361-cf995598345b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tdvz7\" (UID: \"3374d1d7-08e1-409a-8361-cf995598345b\") " pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:31.859076 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.859057 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:31.861190 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:31.861177 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tdvz7" Oct 29 22:33:32.194647 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:32.194558 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vml7h"] Oct 29 22:33:32.196008 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:32.195924 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tdvz7"] Oct 29 22:33:32.197953 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:32.197927 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa7fc2d_d350_4aba_977b_32868a6e3fe7.slice/crio-78d6b541465cb3a513aabe385e34de9e4ffd12a3dbc7f2a7b3415871ec137fd4 WatchSource:0}: Error finding container 78d6b541465cb3a513aabe385e34de9e4ffd12a3dbc7f2a7b3415871ec137fd4: Status 404 returned error can't find the container with id 78d6b541465cb3a513aabe385e34de9e4ffd12a3dbc7f2a7b3415871ec137fd4 Oct 29 22:33:32.203823 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:32.203789 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3374d1d7_08e1_409a_8361_cf995598345b.slice/crio-7bfa9ac68b7df4bf21be0e06aa29560fe9e5552209c65d93c07f5420efe52d07 WatchSource:0}: Error finding container 7bfa9ac68b7df4bf21be0e06aa29560fe9e5552209c65d93c07f5420efe52d07: Status 404 returned error can't find the container with id 7bfa9ac68b7df4bf21be0e06aa29560fe9e5552209c65d93c07f5420efe52d07 Oct 29 22:33:33.177794 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:33.177713 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdvz7" event={"ID":"3374d1d7-08e1-409a-8361-cf995598345b","Type":"ContainerStarted","Data":"5e6f50e4c5417357929d5ab1a0fee1457ca325b33352e3af7db8d9e72c5913a8"} Oct 29 22:33:33.177794 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:33.177784 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdvz7" event={"ID":"3374d1d7-08e1-409a-8361-cf995598345b","Type":"ContainerStarted","Data":"13a5bfd82cd89536652a24e09de22f3a4dc01e3fb43f303d4cd8f52e290be682"} Oct 29 22:33:33.178031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:33.177813 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdvz7" event={"ID":"3374d1d7-08e1-409a-8361-cf995598345b","Type":"ContainerStarted","Data":"7bfa9ac68b7df4bf21be0e06aa29560fe9e5552209c65d93c07f5420efe52d07"} Oct 29 22:33:33.179053 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:33.179029 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vml7h" event={"ID":"efa7fc2d-d350-4aba-977b-32868a6e3fe7","Type":"ContainerStarted","Data":"78d6b541465cb3a513aabe385e34de9e4ffd12a3dbc7f2a7b3415871ec137fd4"} Oct 29 22:33:34.185968 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:34.185934 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vml7h" event={"ID":"efa7fc2d-d350-4aba-977b-32868a6e3fe7","Type":"ContainerStarted","Data":"5df801e95b5f261debe58fb6a2f7265561ed38ffbff05ccf05f910625ca32e4c"} Oct 29 22:33:34.186331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:34.186011 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vml7h" event={"ID":"efa7fc2d-d350-4aba-977b-32868a6e3fe7","Type":"ContainerStarted","Data":"0b25a2427e6002f72f7ff07020976f53d61d3844102515f88519f1360ec4a97a"} Oct 29 22:33:34.186331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:34.186064 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:35.191897 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:35.191863 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tdvz7" event={"ID":"3374d1d7-08e1-409a-8361-cf995598345b","Type":"ContainerStarted","Data":"2e9f5bcb8e6a3ef8cb6a9ad6a607173c1071a88f9fd7ee78d6def37b05a443f6"} Oct 29 22:33:35.584459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:35.584430 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:35.584609 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:35.584546 5030 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 22:33:35.585773 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:35.585756 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b6bb75897-sr2f4" Oct 29 22:33:35.792650 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:33:35.792620 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:33:36.037576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.037549 5030 scope.go:122] "RemoveContainer" containerID="80aeaa22340ccbbc72b8de4c6f7d07b9851aa16491010423826bf11813a06e03" Oct 29 22:33:36.197396 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.197370 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:33:36.197742 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.197485 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" event={"ID":"5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb","Type":"ContainerStarted","Data":"412e0f2326ce2abab71915ecec960e50b6b5c0ab60273de715a213b731d8bc47"} Oct 29 22:33:36.198050 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.198033 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:36.198949 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.198926 5030 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-m855q container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body= Oct 29 22:33:36.199058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.198962 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" podUID="5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" Oct 29 22:33:36.901195 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:36.901167 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:37.001666 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.001632 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:37.065716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.065675 5030 patch_prober.go:29] interesting pod/image-registry-66bf65fd8-phdn2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 29 22:33:37.065894 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.065719 5030 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 22:33:37.105597 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.105562 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:37.203014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.202926 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:37.205644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.205628 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-m855q" Oct 29 22:33:37.282340 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:37.282299 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:38.711148 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:38.711113 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:38.803013 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:38.802981 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:39.153328 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:39.153298 5030 patch_prober.go:29] interesting pod/image-registry-7fdff46b97-drfgt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 29 22:33:39.153492 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:39.153351 5030 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 22:33:40.111807 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.111769 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-hj5vq-jp65r"] Oct 29 22:33:40.116585 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.116566 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.131655 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.130719 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp"] Oct 29 22:33:40.135826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.135807 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-hj5vq-jp65r"] Oct 29 22:33:40.135972 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.135931 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:40.147256 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.147234 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66bf65fd8-phdn2"] Oct 29 22:33:40.154329 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.154303 5030 patch_prober.go:29] interesting pod/image-registry-66bf65fd8-phdn2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 29 22:33:40.154516 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.154493 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 22:33:40.155814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.155773 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp"] Oct 29 22:33:40.157898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.157760 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Oct 29 22:33:40.157898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.157870 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-csz6f\"" Oct 29 22:33:40.222891 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.222866 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-r7wrt"] Oct 29 22:33:40.226112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.226096 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68487bc74d-jbpms"] Oct 29 22:33:40.226231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.226217 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.229029 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.229011 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.240766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.240736 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:40.244422 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.244407 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-r7wrt"] Oct 29 22:33:40.246259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.246238 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 29 22:33:40.247779 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.247756 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68487bc74d-jbpms"] Oct 29 22:33:40.247882 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.247868 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 29 22:33:40.257652 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.257635 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-54g92\"" Oct 29 22:33:40.312116 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.312085 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp5nx\" (UniqueName: \"kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.312281 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.312148 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.312433 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.312411 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.312521 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.312470 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/07cb55c7-98cf-4c00-8e65-11a26ad2eca7-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-hmmgp\" (UID: \"07cb55c7-98cf-4c00-8e65-11a26ad2eca7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:40.312521 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.312513 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.337541 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.337514 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:40.413412 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413362 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.413412 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413393 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/07cb55c7-98cf-4c00-8e65-11a26ad2eca7-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-hmmgp\" (UID: \"07cb55c7-98cf-4c00-8e65-11a26ad2eca7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:40.413549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413421 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-ca-trust-extracted\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.413549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413445 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-installation-pull-secrets\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.413630 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413580 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.413684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413626 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-registry-certificates\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.413684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413656 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp5nx\" (UniqueName: \"kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.413781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413727 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.413781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413763 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg4qc\" (UniqueName: \"kubernetes.io/projected/3897b32b-ba8f-41b7-bc91-db7901bb8335-kube-api-access-bg4qc\") pod \"downloads-6dd789c8b7-r7wrt\" (UID: \"3897b32b-ba8f-41b7-bc91-db7901bb8335\") " pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.413924 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413794 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-trusted-ca\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.413924 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413817 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-bound-sa-token\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.414030 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.413920 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-image-registry-private-configuration\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.414030 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.414010 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xqz\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-kube-api-access-v4xqz\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.414189 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.414062 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-registry-tls\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.414189 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.414093 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3897b32b-ba8f-41b7-bc91-db7901bb8335-tmp\") pod \"downloads-6dd789c8b7-r7wrt\" (UID: \"3897b32b-ba8f-41b7-bc91-db7901bb8335\") " pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.414266 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.414229 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.414356 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.414340 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.414401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.414358 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.415961 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.415944 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/07cb55c7-98cf-4c00-8e65-11a26ad2eca7-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-hmmgp\" (UID: \"07cb55c7-98cf-4c00-8e65-11a26ad2eca7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:40.434942 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.434920 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp5nx\" (UniqueName: \"kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx\") pod \"periodic-gathering-hj5vq-jp65r\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.435667 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.435651 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:33:40.439815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.439726 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:40.450682 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.450665 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:40.515622 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515597 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-ca-trust-extracted\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.515741 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515636 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-installation-pull-secrets\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.515741 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515682 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-registry-certificates\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.515921 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515750 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg4qc\" (UniqueName: \"kubernetes.io/projected/3897b32b-ba8f-41b7-bc91-db7901bb8335-kube-api-access-bg4qc\") pod \"downloads-6dd789c8b7-r7wrt\" (UID: \"3897b32b-ba8f-41b7-bc91-db7901bb8335\") " pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.515921 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515778 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-trusted-ca\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.515921 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515801 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-bound-sa-token\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.515921 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515833 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-image-registry-private-configuration\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.515921 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515888 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xqz\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-kube-api-access-v4xqz\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.516166 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515933 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-registry-tls\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.516166 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.515965 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3897b32b-ba8f-41b7-bc91-db7901bb8335-tmp\") pod \"downloads-6dd789c8b7-r7wrt\" (UID: \"3897b32b-ba8f-41b7-bc91-db7901bb8335\") " pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.516384 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.516364 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3897b32b-ba8f-41b7-bc91-db7901bb8335-tmp\") pod \"downloads-6dd789c8b7-r7wrt\" (UID: \"3897b32b-ba8f-41b7-bc91-db7901bb8335\") " pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.518347 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.516700 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-ca-trust-extracted\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.518347 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.518306 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-registry-certificates\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.520044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.519616 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-trusted-ca\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.521781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.521495 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-registry-tls\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.523037 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.523015 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-image-registry-private-configuration\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.523232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.523213 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-installation-pull-secrets\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.538529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.538505 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-bound-sa-token\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.538626 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.538563 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xqz\" (UniqueName: \"kubernetes.io/projected/fe2799d4-b34d-47a7-acb8-80ffbf92b66b-kube-api-access-v4xqz\") pod \"image-registry-68487bc74d-jbpms\" (UID: \"fe2799d4-b34d-47a7-acb8-80ffbf92b66b\") " pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.538626 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.538583 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg4qc\" (UniqueName: \"kubernetes.io/projected/3897b32b-ba8f-41b7-bc91-db7901bb8335-kube-api-access-bg4qc\") pod \"downloads-6dd789c8b7-r7wrt\" (UID: \"3897b32b-ba8f-41b7-bc91-db7901bb8335\") " pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.544615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.544447 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:40.738817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.738756 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-hj5vq-jp65r"] Oct 29 22:33:40.742088 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:40.742060 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb7b98b_4661_47a2_8c64_97e7efc2d381.slice/crio-eae3bbb64cbb55f368d2ed4aedb47589a7df78b121430de50ba8076d2d9a7a50 WatchSource:0}: Error finding container eae3bbb64cbb55f368d2ed4aedb47589a7df78b121430de50ba8076d2d9a7a50: Status 404 returned error can't find the container with id eae3bbb64cbb55f368d2ed4aedb47589a7df78b121430de50ba8076d2d9a7a50 Oct 29 22:33:40.762204 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.762178 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp"] Oct 29 22:33:40.769864 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:40.769805 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07cb55c7_98cf_4c00_8e65_11a26ad2eca7.slice/crio-6325e13ca1cb853feb156b6a105b002f36faa893cd1e03d4207b356ccc3a047e WatchSource:0}: Error finding container 6325e13ca1cb853feb156b6a105b002f36faa893cd1e03d4207b356ccc3a047e: Status 404 returned error can't find the container with id 6325e13ca1cb853feb156b6a105b002f36faa893cd1e03d4207b356ccc3a047e Oct 29 22:33:40.838414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.838386 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:33:40.842320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:40.842294 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68487bc74d-jbpms"] Oct 29 22:33:40.848648 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:40.848621 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2799d4_b34d_47a7_acb8_80ffbf92b66b.slice/crio-3064b7db6d6e01d00023bf9f5323d3c0652b1b619babfd202f607e56a213b71d WatchSource:0}: Error finding container 3064b7db6d6e01d00023bf9f5323d3c0652b1b619babfd202f607e56a213b71d: Status 404 returned error can't find the container with id 3064b7db6d6e01d00023bf9f5323d3c0652b1b619babfd202f607e56a213b71d Oct 29 22:33:41.140804 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.140731 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-r7wrt"] Oct 29 22:33:41.143476 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:41.143439 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3897b32b_ba8f_41b7_bc91_db7901bb8335.slice/crio-880fd136a803f1363c0a77d2a23a9b55e8abaec9c481040ee4e977264a896974 WatchSource:0}: Error finding container 880fd136a803f1363c0a77d2a23a9b55e8abaec9c481040ee4e977264a896974: Status 404 returned error can't find the container with id 880fd136a803f1363c0a77d2a23a9b55e8abaec9c481040ee4e977264a896974 Oct 29 22:33:41.220214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.220167 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" event={"ID":"07cb55c7-98cf-4c00-8e65-11a26ad2eca7","Type":"ContainerStarted","Data":"6325e13ca1cb853feb156b6a105b002f36faa893cd1e03d4207b356ccc3a047e"} Oct 29 22:33:41.222393 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.222366 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" event={"ID":"fe2799d4-b34d-47a7-acb8-80ffbf92b66b","Type":"ContainerStarted","Data":"2b34d015e28e653681ad3369464a10b47370cdc631817d40b0378bc19318c798"} Oct 29 22:33:41.222525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.222409 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" event={"ID":"fe2799d4-b34d-47a7-acb8-80ffbf92b66b","Type":"ContainerStarted","Data":"3064b7db6d6e01d00023bf9f5323d3c0652b1b619babfd202f607e56a213b71d"} Oct 29 22:33:41.222525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.222452 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:33:41.226220 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.225876 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" event={"ID":"9fb7b98b-4661-47a2-8c64-97e7efc2d381","Type":"ContainerStarted","Data":"5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772"} Oct 29 22:33:41.226220 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.225916 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" event={"ID":"9fb7b98b-4661-47a2-8c64-97e7efc2d381","Type":"ContainerStarted","Data":"eae3bbb64cbb55f368d2ed4aedb47589a7df78b121430de50ba8076d2d9a7a50"} Oct 29 22:33:41.227532 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.227510 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-r7wrt" event={"ID":"3897b32b-ba8f-41b7-bc91-db7901bb8335","Type":"ContainerStarted","Data":"880fd136a803f1363c0a77d2a23a9b55e8abaec9c481040ee4e977264a896974"} Oct 29 22:33:41.267731 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.267676 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" podStartSLOduration=1.267658395 podStartE2EDuration="1.267658395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:41.266484936 +0000 UTC m=+18.379130551" watchObservedRunningTime="2026-10-29 22:33:41.267658395 +0000 UTC m=+18.380303975" Oct 29 22:33:41.554020 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.553975 5030 patch_prober.go:29] interesting pod/image-registry-7fdff46b97-drfgt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 29 22:33:41.554233 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.554035 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 22:33:41.789282 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.789257 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vml7h_efa7fc2d-d350-4aba-977b-32868a6e3fe7/dns/0.log" Oct 29 22:33:41.989581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:41.989551 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vml7h_efa7fc2d-d350-4aba-977b-32868a6e3fe7/kube-rbac-proxy/0.log" Oct 29 22:33:42.189244 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.189218 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9655m_1ac5c5e2-a666-4cbf-9a73-677929dc640e/dns-node-resolver/0.log" Oct 29 22:33:42.234171 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.234137 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" event={"ID":"07cb55c7-98cf-4c00-8e65-11a26ad2eca7","Type":"ContainerStarted","Data":"1d6bf6554556174f7001355e29869cb4fb356c74c80d56d8cc5f5bc3632e264a"} Oct 29 22:33:42.234350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.234311 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:42.240118 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.240055 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" Oct 29 22:33:42.275751 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.275692 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-hmmgp" podStartSLOduration=1.32745417 podStartE2EDuration="2.275676837s" totalImagesPullingTime="948.222667ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:42.275131065 +0000 UTC m=+19.387776677" watchObservedRunningTime="2026-10-29 22:33:42.275676837 +0000 UTC m=+19.388322425" Oct 29 22:33:42.276684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.276649 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" podStartSLOduration=2.276639057 podStartE2EDuration="2.276639057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:41.308970844 +0000 UTC m=+18.421616433" watchObservedRunningTime="2026-10-29 22:33:42.276639057 +0000 UTC m=+19.389284646" Oct 29 22:33:42.389853 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.389797 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66bf65fd8-phdn2_eea9c729-9323-4535-bb3b-f5e67bb28b29/registry/0.log" Oct 29 22:33:42.465034 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.464997 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:42.565348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.565265 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:42.589651 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.589622 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68487bc74d-jbpms_fe2799d4-b34d-47a7-acb8-80ffbf92b66b/registry/0.log" Oct 29 22:33:42.665624 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.665549 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:42.767855 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.767811 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:42.791613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.791584 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl"] Oct 29 22:33:42.795472 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.795437 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:42.795598 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.795443 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7fdff46b97-drfgt_05dc76a9-2747-4bd0-907d-a9b2caa3b25d/registry/0.log" Oct 29 22:33:42.811960 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.811936 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl"] Oct 29 22:33:42.825164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.825107 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 29 22:33:42.825164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.825123 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-q69jv\"" Oct 29 22:33:42.825335 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.825203 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 29 22:33:42.826551 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.826536 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 29 22:33:42.853750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.853704 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:42.933605 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.933573 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3d52fe7-a39c-43de-bf41-ee90256e755a-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:42.933784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.933625 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzw9p\" (UniqueName: \"kubernetes.io/projected/b3d52fe7-a39c-43de-bf41-ee90256e755a-kube-api-access-pzw9p\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:42.933784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.933771 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3d52fe7-a39c-43de-bf41-ee90256e755a-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:42.933932 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.933810 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3d52fe7-a39c-43de-bf41-ee90256e755a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:42.989506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:42.989482 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkr9j_c7881bd2-9f24-4c58-9a77-68cc091ee9ab/node-ca/0.log" Oct 29 22:33:43.035063 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.035020 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3d52fe7-a39c-43de-bf41-ee90256e755a-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.035213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.035077 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw9p\" (UniqueName: \"kubernetes.io/projected/b3d52fe7-a39c-43de-bf41-ee90256e755a-kube-api-access-pzw9p\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.035213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.035148 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3d52fe7-a39c-43de-bf41-ee90256e755a-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.035213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.035185 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3d52fe7-a39c-43de-bf41-ee90256e755a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.036154 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.036092 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3d52fe7-a39c-43de-bf41-ee90256e755a-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.040302 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.040272 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3d52fe7-a39c-43de-bf41-ee90256e755a-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.044197 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.044178 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3d52fe7-a39c-43de-bf41-ee90256e755a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.056546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.056517 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzw9p\" (UniqueName: \"kubernetes.io/projected/b3d52fe7-a39c-43de-bf41-ee90256e755a-kube-api-access-pzw9p\") pod \"prometheus-operator-9cd89b74c-pnvfl\" (UID: \"b3d52fe7-a39c-43de-bf41-ee90256e755a\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.113608 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.113556 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" Oct 29 22:33:43.190895 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.190815 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6bb75897-sr2f4_e9fb3bd9-7312-4616-a8db-6e51b56d78a2/router/0.log" Oct 29 22:33:43.389191 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.389109 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrr8p_8db65a63-2853-43db-8877-83a0d26c2de2/serve-healthcheck-canary/0.log" Oct 29 22:33:43.423896 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.423868 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl"] Oct 29 22:33:43.429998 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:43.429970 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d52fe7_a39c_43de_bf41_ee90256e755a.slice/crio-68eaa7006c11a141698ef6df9a06aec8b0abf0b9421b44c1e650c520c0d91763 WatchSource:0}: Error finding container 68eaa7006c11a141698ef6df9a06aec8b0abf0b9421b44c1e650c520c0d91763: Status 404 returned error can't find the container with id 68eaa7006c11a141698ef6df9a06aec8b0abf0b9421b44c1e650c520c0d91763 Oct 29 22:33:43.964048 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:43.964019 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-skdsx" Oct 29 22:33:44.194735 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:44.194708 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vml7h" Oct 29 22:33:44.247548 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:44.247517 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" event={"ID":"b3d52fe7-a39c-43de-bf41-ee90256e755a","Type":"ContainerStarted","Data":"68eaa7006c11a141698ef6df9a06aec8b0abf0b9421b44c1e650c520c0d91763"} Oct 29 22:33:45.254375 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:45.254338 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" event={"ID":"b3d52fe7-a39c-43de-bf41-ee90256e755a","Type":"ContainerStarted","Data":"43264c4a2099e9932bf0cdfddae816b246a0d850f7e39d33692cdbbcddda1871"} Oct 29 22:33:45.254804 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:45.254411 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" event={"ID":"b3d52fe7-a39c-43de-bf41-ee90256e755a","Type":"ContainerStarted","Data":"57ee9338bd55d228864174783bb7f91c5695180d21c07562283f1a0645b56dd8"} Oct 29 22:33:45.302109 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:45.301031 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-pnvfl" podStartSLOduration=2.152274946 podStartE2EDuration="3.301006193s" totalImagesPullingTime="1.148731247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:45.298905284 +0000 UTC m=+22.411550873" watchObservedRunningTime="2026-10-29 22:33:45.301006193 +0000 UTC m=+22.413651784" Oct 29 22:33:45.431061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:45.430974 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.120730 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.120698 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.292285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.292249 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.384782 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.384709 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb"] Oct 29 22:33:47.394112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.394078 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.426947 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.426913 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb"] Oct 29 22:33:47.427094 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.427047 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.438667 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.438640 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms"] Oct 29 22:33:47.452530 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.452489 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-prfdr\"" Oct 29 22:33:47.452530 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.452499 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Oct 29 22:33:47.452530 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.452523 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Oct 29 22:33:47.461497 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.461474 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5fvq\" (UniqueName: \"kubernetes.io/projected/0fd020fd-242a-4233-885d-c120ad267ce2-kube-api-access-t5fvq\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.461623 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.461507 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fd020fd-242a-4233-885d-c120ad267ce2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.461623 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.461567 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fd020fd-242a-4233-885d-c120ad267ce2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.461697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.461667 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fd020fd-242a-4233-885d-c120ad267ce2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.462628 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.462607 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v7bzc"] Oct 29 22:33:47.462766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.462743 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.479419 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.479400 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms"] Oct 29 22:33:47.479529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.479515 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.485289 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.485250 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 29 22:33:47.490292 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.490268 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.496084 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.496065 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 29 22:33:47.530737 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.530703 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-k6rg6\"" Oct 29 22:33:47.530885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.530806 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kfh6g\"" Oct 29 22:33:47.530885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.530881 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 29 22:33:47.531002 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.530940 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 29 22:33:47.531002 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.530957 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 29 22:33:47.531082 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.531005 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 29 22:33:47.562574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562547 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-wtmp\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.562667 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562577 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03802761-7c3b-41b3-a7c5-81f4f225527e-metrics-client-ca\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.562667 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562610 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fd020fd-242a-4233-885d-c120ad267ce2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.562803 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562788 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.562901 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562818 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-textfile\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.562955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562900 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-accelerators-collector-config\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.562955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562937 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fd020fd-242a-4233-885d-c120ad267ce2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.563060 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.562989 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.563060 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563041 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-sys\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.563159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563074 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx7zs\" (UniqueName: \"kubernetes.io/projected/03802761-7c3b-41b3-a7c5-81f4f225527e-kube-api-access-kx7zs\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.563159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563103 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/48d3d9f0-9c5e-4296-bb37-2bee105913c7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.563159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563147 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-tls\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.563305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563181 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48d3d9f0-9c5e-4296-bb37-2bee105913c7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.563305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563205 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxkn6\" (UniqueName: \"kubernetes.io/projected/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-api-access-wxkn6\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.563305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563244 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5fvq\" (UniqueName: \"kubernetes.io/projected/0fd020fd-242a-4233-885d-c120ad267ce2-kube-api-access-t5fvq\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.563305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563270 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.563305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563299 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.563544 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563340 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fd020fd-242a-4233-885d-c120ad267ce2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.563544 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563343 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fd020fd-242a-4233-885d-c120ad267ce2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.563544 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.563369 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-root\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.566252 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.566230 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fd020fd-242a-4233-885d-c120ad267ce2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.566355 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.566299 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fd020fd-242a-4233-885d-c120ad267ce2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.585345 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.585325 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5fvq\" (UniqueName: \"kubernetes.io/projected/0fd020fd-242a-4233-885d-c120ad267ce2-kube-api-access-t5fvq\") pod \"openshift-state-metrics-7bd7489957-6gbxb\" (UID: \"0fd020fd-242a-4233-885d-c120ad267ce2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.602445 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.602413 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.664002 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.663943 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.664002 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.663980 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-textfile\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664002 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.663999 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-accelerators-collector-config\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664224 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664206 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.664284 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664257 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-sys\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664291 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx7zs\" (UniqueName: \"kubernetes.io/projected/03802761-7c3b-41b3-a7c5-81f4f225527e-kube-api-access-kx7zs\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664321 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/48d3d9f0-9c5e-4296-bb37-2bee105913c7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.664442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664345 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-sys\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664347 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-textfile\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664392 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-tls\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664427 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48d3d9f0-9c5e-4296-bb37-2bee105913c7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664451 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkn6\" (UniqueName: \"kubernetes.io/projected/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-api-access-wxkn6\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664490 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664516 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664560 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-root\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664598 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-wtmp\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664624 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03802761-7c3b-41b3-a7c5-81f4f225527e-metrics-client-ca\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.664637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664626 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-accelerators-collector-config\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.665016 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664641 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/48d3d9f0-9c5e-4296-bb37-2bee105913c7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.665016 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664807 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.665016 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.664903 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-root\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.665179 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.665155 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03802761-7c3b-41b3-a7c5-81f4f225527e-metrics-client-ca\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.665224 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.665188 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-wtmp\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.665915 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.665867 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48d3d9f0-9c5e-4296-bb37-2bee105913c7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.667108 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.667071 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.667375 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.667352 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.667672 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.667644 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/03802761-7c3b-41b3-a7c5-81f4f225527e-node-exporter-tls\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.668057 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.668026 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.686500 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.686475 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx7zs\" (UniqueName: \"kubernetes.io/projected/03802761-7c3b-41b3-a7c5-81f4f225527e-kube-api-access-kx7zs\") pod \"node-exporter-v7bzc\" (UID: \"03802761-7c3b-41b3-a7c5-81f4f225527e\") " pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.687151 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.687129 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxkn6\" (UniqueName: \"kubernetes.io/projected/48d3d9f0-9c5e-4296-bb37-2bee105913c7-kube-api-access-wxkn6\") pod \"kube-state-metrics-5dc8d7d89b-75qms\" (UID: \"48d3d9f0-9c5e-4296-bb37-2bee105913c7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.719892 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.719858 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:47.796971 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.796940 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" Oct 29 22:33:47.906407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.906183 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v7bzc" Oct 29 22:33:47.928450 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.928385 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" Oct 29 22:33:47.949910 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:47.949873 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03802761_7c3b_41b3_a7c5_81f4f225527e.slice/crio-2aafb88ff4c5e92f3e96ef74a288b3ab8c598f53ec258f02c00fc3a4eb9ff967 WatchSource:0}: Error finding container 2aafb88ff4c5e92f3e96ef74a288b3ab8c598f53ec258f02c00fc3a4eb9ff967: Status 404 returned error can't find the container with id 2aafb88ff4c5e92f3e96ef74a288b3ab8c598f53ec258f02c00fc3a4eb9ff967 Oct 29 22:33:47.974665 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:47.974633 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:48.136225 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.136195 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb"] Oct 29 22:33:48.272322 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.272287 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7bzc" event={"ID":"03802761-7c3b-41b3-a7c5-81f4f225527e","Type":"ContainerStarted","Data":"2aafb88ff4c5e92f3e96ef74a288b3ab8c598f53ec258f02c00fc3a4eb9ff967"} Oct 29 22:33:48.275770 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.275747 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" event={"ID":"0fd020fd-242a-4233-885d-c120ad267ce2","Type":"ContainerStarted","Data":"efe24bb948898fa755ce344f2d9f398e4abd412abe54a74f2c9bb85fe7e6af6a"} Oct 29 22:33:48.275937 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.275808 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" event={"ID":"0fd020fd-242a-4233-885d-c120ad267ce2","Type":"ContainerStarted","Data":"44a08c9a6d239323bbadf4625321bceb8a477584baa6b5bdae04de552bda8813"} Oct 29 22:33:48.291195 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.291168 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms"] Oct 29 22:33:48.304779 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:48.304741 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d3d9f0_9c5e_4296_bb37_2bee105913c7.slice/crio-203384a0f7ad590b1d643d42396a2ac4ad7c576254d0b73cdcaa057cbec665ce WatchSource:0}: Error finding container 203384a0f7ad590b1d643d42396a2ac4ad7c576254d0b73cdcaa057cbec665ce: Status 404 returned error can't find the container with id 203384a0f7ad590b1d643d42396a2ac4ad7c576254d0b73cdcaa057cbec665ce Oct 29 22:33:48.524268 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.524217 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:48.678825 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.678772 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:48.818109 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.817962 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:48.818775 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.818750 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:33:48.841693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.841662 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.846043 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.845954 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:33:48.858738 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.858693 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 29 22:33:48.859586 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859564 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 29 22:33:48.859706 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859618 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 29 22:33:48.859770 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859736 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-54n2z\"" Oct 29 22:33:48.859826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859776 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 29 22:33:48.859917 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859856 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 29 22:33:48.859964 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859916 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 29 22:33:48.859964 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.859949 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 29 22:33:48.883273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883227 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883273 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2cbr\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883313 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883341 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883372 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883399 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883470 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883505 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883953 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883554 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883953 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883582 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883953 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883607 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883953 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883628 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.883953 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.883660 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.956565 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.956532 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb79dfbcb-ctbr6"] Oct 29 22:33:48.960433 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.960403 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 29 22:33:48.960563 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.960454 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 29 22:33:48.973864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.973820 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.979281 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.977395 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb79dfbcb-ctbr6"] Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984488 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984532 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984564 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984597 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984623 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984663 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984689 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984713 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984736 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984757 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984787 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984822 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984864 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984888 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2cbr\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984930 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984965 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.984994 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4cqf\" (UniqueName: \"kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.985027 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.985058 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.985784 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.985087 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.991939 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.987407 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.991939 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.988763 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:48.998030 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.997724 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 29 22:33:48.998030 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.997960 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 29 22:33:48.998142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.998067 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 29 22:33:48.999189 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:48.999166 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.001307 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.001285 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.004256 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.004229 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.007151 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.007108 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.007434 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.007409 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.007524 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.007457 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.008019 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.007991 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.008399 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.008378 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.008881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.008809 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.009624 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.009604 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.009723 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.009684 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2cbr\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr\") pod \"alertmanager-main-0\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.016089 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.015923 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:33:49.060459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.060266 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 29 22:33:49.060459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.060349 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4952v\"" Oct 29 22:33:49.060459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.060425 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 29 22:33:49.087325 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087260 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.087449 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087339 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.087449 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087407 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.087521 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087453 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4cqf\" (UniqueName: \"kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.087576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087563 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.087608 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087594 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.087663 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.087647 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.088411 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.088384 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.090077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.089800 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.090077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.089806 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.090615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.090520 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.092534 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.092511 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.101207 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.101184 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.112196 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.112149 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4cqf\" (UniqueName: \"kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf\") pod \"console-5cb79dfbcb-ctbr6\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.287713 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.287676 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" event={"ID":"0fd020fd-242a-4233-885d-c120ad267ce2","Type":"ContainerStarted","Data":"4cc7d95b4735c32c1b3f0353a43b1e6bbcf1ba01c37f368afdf2abf7166124fa"} Oct 29 22:33:49.289665 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.289629 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" event={"ID":"48d3d9f0-9c5e-4296-bb37-2bee105913c7","Type":"ContainerStarted","Data":"203384a0f7ad590b1d643d42396a2ac4ad7c576254d0b73cdcaa057cbec665ce"} Oct 29 22:33:49.336204 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.335930 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:49.373782 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.373753 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:33:49.390312 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:49.390269 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d90f791_07a7_47a8_8ba1_07b424c37404.slice/crio-ded61dc51bb5638feaa7e38bbba6f77a3e7259a7c8f4581e4af5d6c8a3d29fb3 WatchSource:0}: Error finding container ded61dc51bb5638feaa7e38bbba6f77a3e7259a7c8f4581e4af5d6c8a3d29fb3: Status 404 returned error can't find the container with id ded61dc51bb5638feaa7e38bbba6f77a3e7259a7c8f4581e4af5d6c8a3d29fb3 Oct 29 22:33:49.619544 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.619466 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7986f44ccf-p28v2"] Oct 29 22:33:49.655861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.655797 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7986f44ccf-p28v2"] Oct 29 22:33:49.656009 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.655960 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.673085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673058 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb79dfbcb-ctbr6"] Oct 29 22:33:49.673341 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673321 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9slvr\"" Oct 29 22:33:49.673453 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673415 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Oct 29 22:33:49.673540 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673526 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Oct 29 22:33:49.673685 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673671 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Oct 29 22:33:49.673777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673743 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Oct 29 22:33:49.673777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673742 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d41itj5shn6c3\"" Oct 29 22:33:49.673777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.673770 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Oct 29 22:33:49.679416 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:49.678587 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f89320_c805_4ff5_b530_6bb70cd7169d.slice/crio-879b5961684ff8f26633e801d6292f5fcbe33ffc9956b22365e769eb036c836d WatchSource:0}: Error finding container 879b5961684ff8f26633e801d6292f5fcbe33ffc9956b22365e769eb036c836d: Status 404 returned error can't find the container with id 879b5961684ff8f26633e801d6292f5fcbe33ffc9956b22365e769eb036c836d Oct 29 22:33:49.693214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693186 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvprh\" (UniqueName: \"kubernetes.io/projected/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-kube-api-access-hvprh\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693344 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693230 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-tls\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693344 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693281 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693344 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693336 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-metrics-client-ca\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693404 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693428 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-tmp\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693451 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-grpc-tls\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693500 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.693632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.693563 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.797750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.797713 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-metrics-client-ca\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.797903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.797793 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.797903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.797821 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-tmp\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.797903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.797867 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-grpc-tls\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.798072 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.797915 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.798072 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.797971 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.798072 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.798006 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvprh\" (UniqueName: \"kubernetes.io/projected/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-kube-api-access-hvprh\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.798072 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.798038 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-tls\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.798249 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.798086 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.803011 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.802973 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-tmp\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.803377 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.803354 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-metrics-client-ca\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.807755 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.807726 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.807899 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.807885 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.808124 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.808097 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-grpc-tls\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.808583 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.808556 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-tls\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.808832 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.808814 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.809047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.809023 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.824879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.824825 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvprh\" (UniqueName: \"kubernetes.io/projected/ddd8a66a-5b7f-436e-923f-3a69a9ee546f-kube-api-access-hvprh\") pod \"thanos-querier-7986f44ccf-p28v2\" (UID: \"ddd8a66a-5b7f-436e-923f-3a69a9ee546f\") " pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:49.830089 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:49.830056 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:50.090259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.090225 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:50.154100 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.154073 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:33:50.176389 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.176308 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7986f44ccf-p28v2"] Oct 29 22:33:50.177772 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.177743 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:50.226861 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:50.226798 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd8a66a_5b7f_436e_923f_3a69a9ee546f.slice/crio-1f430d1cf9ee2c0c05139c50f822081fa9b1ddf4e7595483104db465ce963731 WatchSource:0}: Error finding container 1f430d1cf9ee2c0c05139c50f822081fa9b1ddf4e7595483104db465ce963731: Status 404 returned error can't find the container with id 1f430d1cf9ee2c0c05139c50f822081fa9b1ddf4e7595483104db465ce963731 Oct 29 22:33:50.297228 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.297201 5030 generic.go:356] "Generic (PLEG): container finished" podID="03802761-7c3b-41b3-a7c5-81f4f225527e" containerID="0bd5f837ac6b714ef373d0ed01d19a6f8793791794268cfbbc54ab2590ff2bfc" exitCode=0 Oct 29 22:33:50.297630 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.297331 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7bzc" event={"ID":"03802761-7c3b-41b3-a7c5-81f4f225527e","Type":"ContainerDied","Data":"0bd5f837ac6b714ef373d0ed01d19a6f8793791794268cfbbc54ab2590ff2bfc"} Oct 29 22:33:50.299437 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.299414 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"1f430d1cf9ee2c0c05139c50f822081fa9b1ddf4e7595483104db465ce963731"} Oct 29 22:33:50.301016 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.300947 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb79dfbcb-ctbr6" event={"ID":"37f89320-c805-4ff5-b530-6bb70cd7169d","Type":"ContainerStarted","Data":"879b5961684ff8f26633e801d6292f5fcbe33ffc9956b22365e769eb036c836d"} Oct 29 22:33:50.302479 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.302457 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"ded61dc51bb5638feaa7e38bbba6f77a3e7259a7c8f4581e4af5d6c8a3d29fb3"} Oct 29 22:33:50.585737 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.585631 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:50.686897 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.686773 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:50.789864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.789720 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:50.895818 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:50.895742 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:51.093547 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.093510 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:51.131771 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.131743 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rh76g" Oct 29 22:33:51.311140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.311106 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" event={"ID":"48d3d9f0-9c5e-4296-bb37-2bee105913c7","Type":"ContainerStarted","Data":"371a5ea370db3bd88ffce258edbb520f9bec464485a15dc7e67f970c0235a49f"} Oct 29 22:33:51.311533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.311191 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" event={"ID":"48d3d9f0-9c5e-4296-bb37-2bee105913c7","Type":"ContainerStarted","Data":"8aee8b9ac41db4a55faed9a8c058bcb2ba63e8340bfd8787c38c4d1b5c67baeb"} Oct 29 22:33:51.311533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.311223 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" event={"ID":"48d3d9f0-9c5e-4296-bb37-2bee105913c7","Type":"ContainerStarted","Data":"6cb0faedf7a4884381022350259d2f3e80b325de28600e67f4fc7100f52906f2"} Oct 29 22:33:51.314277 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.314251 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7bzc" event={"ID":"03802761-7c3b-41b3-a7c5-81f4f225527e","Type":"ContainerStarted","Data":"a33031ba7c37bc5f2a796fecc2d270c8e3ee96e5c79c9c62c339f666389f3899"} Oct 29 22:33:51.314400 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.314301 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v7bzc" event={"ID":"03802761-7c3b-41b3-a7c5-81f4f225527e","Type":"ContainerStarted","Data":"80b6426dd03a8a5b095af9bbdff51bf612073023fc573333b8ec9ced6a9a66ab"} Oct 29 22:33:51.318230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.318204 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" event={"ID":"0fd020fd-242a-4233-885d-c120ad267ce2","Type":"ContainerStarted","Data":"76563f277bb591036925b92bfb5630f7443641ba93be86f839cb098e71db2189"} Oct 29 22:33:51.358031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.357982 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-75qms" podStartSLOduration=2.405331414 podStartE2EDuration="4.357966507s" totalImagesPullingTime="1.952635093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:51.357409349 +0000 UTC m=+28.470054941" watchObservedRunningTime="2026-10-29 22:33:51.357966507 +0000 UTC m=+28.470612095" Oct 29 22:33:51.450094 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.450019 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v7bzc" podStartSLOduration=2.580462255 podStartE2EDuration="4.449999975s" totalImagesPullingTime="1.763862772s" totalInitContainerRuntime="105.674948ms" isStatefulPod=true podCreationTimestamp="2026-10-29 22:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:51.448656211 +0000 UTC m=+28.561301802" watchObservedRunningTime="2026-10-29 22:33:51.449999975 +0000 UTC m=+28.562645564" Oct 29 22:33:51.450986 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.450941 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-6gbxb" podStartSLOduration=2.086089484 podStartE2EDuration="4.450923351s" totalImagesPullingTime="2.364833867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:51.4043762 +0000 UTC m=+28.517021789" watchObservedRunningTime="2026-10-29 22:33:51.450923351 +0000 UTC m=+28.563568940" Oct 29 22:33:51.553936 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.553908 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:33:51.823140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.823031 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:51.915863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.915606 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-77dd54f899-2qj6z"] Oct 29 22:33:51.930883 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.930710 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:51.952164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.952128 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77dd54f899-2qj6z"] Oct 29 22:33:51.952321 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.952241 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:51.969417 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.969395 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 29 22:33:51.969973 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.969936 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 29 22:33:51.970179 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.969969 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 29 22:33:51.970179 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.970003 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 29 22:33:51.983400 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.983375 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-381j8d2gq0ds8\"" Oct 29 22:33:51.983505 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:51.983440 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9mnjz\"" Oct 29 22:33:52.026156 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026115 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/154474f2-5e45-48f6-9a83-2aa3866e2328-audit-log\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.026291 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026177 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-secret-metrics-server-client-certs\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.026291 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026274 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-secret-metrics-server-tls\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.026397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026319 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154474f2-5e45-48f6-9a83-2aa3866e2328-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.026397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026348 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-client-ca-bundle\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.026397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026366 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/154474f2-5e45-48f6-9a83-2aa3866e2328-metrics-server-audit-profiles\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.026514 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.026428 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4s49\" (UniqueName: \"kubernetes.io/projected/154474f2-5e45-48f6-9a83-2aa3866e2328-kube-api-access-g4s49\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127565 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127489 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/154474f2-5e45-48f6-9a83-2aa3866e2328-audit-log\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127565 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127555 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-secret-metrics-server-client-certs\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127586 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-secret-metrics-server-tls\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127621 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154474f2-5e45-48f6-9a83-2aa3866e2328-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127654 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-client-ca-bundle\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127682 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/154474f2-5e45-48f6-9a83-2aa3866e2328-metrics-server-audit-profiles\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.127823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.127757 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4s49\" (UniqueName: \"kubernetes.io/projected/154474f2-5e45-48f6-9a83-2aa3866e2328-kube-api-access-g4s49\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.128474 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.128449 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/154474f2-5e45-48f6-9a83-2aa3866e2328-audit-log\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.130213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.129461 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154474f2-5e45-48f6-9a83-2aa3866e2328-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.130213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.130161 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/154474f2-5e45-48f6-9a83-2aa3866e2328-metrics-server-audit-profiles\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.131677 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.131652 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-secret-metrics-server-client-certs\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.133546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.133527 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-client-ca-bundle\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.134090 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.134066 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/154474f2-5e45-48f6-9a83-2aa3866e2328-secret-metrics-server-tls\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.151935 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.151892 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4s49\" (UniqueName: \"kubernetes.io/projected/154474f2-5e45-48f6-9a83-2aa3866e2328-kube-api-access-g4s49\") pod \"metrics-server-77dd54f899-2qj6z\" (UID: \"154474f2-5e45-48f6-9a83-2aa3866e2328\") " pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.225882 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.223500 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:52.273963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.273932 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-blqdr"] Oct 29 22:33:52.282171 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.281926 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:33:52.303759 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.303736 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-blqdr"] Oct 29 22:33:52.303938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.303872 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.320399 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.320324 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:52.320776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.320504 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Oct 29 22:33:52.320776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.320533 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-np69l\"" Oct 29 22:33:52.326649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.326625 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568" exitCode=0 Oct 29 22:33:52.326772 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.326759 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568"} Oct 29 22:33:52.329193 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.329171 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86kfb\" (UniqueName: \"kubernetes.io/projected/d39741d3-5f7e-4082-9bc3-a1c5d38447da-kube-api-access-86kfb\") pod \"monitoring-plugin-77f4f76594-blqdr\" (UID: \"d39741d3-5f7e-4082-9bc3-a1c5d38447da\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.329325 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.329213 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d39741d3-5f7e-4082-9bc3-a1c5d38447da-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-blqdr\" (UID: \"d39741d3-5f7e-4082-9bc3-a1c5d38447da\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.430511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.430471 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86kfb\" (UniqueName: \"kubernetes.io/projected/d39741d3-5f7e-4082-9bc3-a1c5d38447da-kube-api-access-86kfb\") pod \"monitoring-plugin-77f4f76594-blqdr\" (UID: \"d39741d3-5f7e-4082-9bc3-a1c5d38447da\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.430668 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.430528 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d39741d3-5f7e-4082-9bc3-a1c5d38447da-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-blqdr\" (UID: \"d39741d3-5f7e-4082-9bc3-a1c5d38447da\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.436877 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.436817 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d39741d3-5f7e-4082-9bc3-a1c5d38447da-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-blqdr\" (UID: \"d39741d3-5f7e-4082-9bc3-a1c5d38447da\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.456217 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.456160 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86kfb\" (UniqueName: \"kubernetes.io/projected/d39741d3-5f7e-4082-9bc3-a1c5d38447da-kube-api-access-86kfb\") pod \"monitoring-plugin-77f4f76594-blqdr\" (UID: \"d39741d3-5f7e-4082-9bc3-a1c5d38447da\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:52.479097 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.479014 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:52.646771 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.646628 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77dd54f899-2qj6z"] Oct 29 22:33:52.682201 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:52.682164 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:53.002067 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:53.002034 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-blqdr"] Oct 29 22:33:53.007319 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:53.007290 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39741d3_5f7e_4082_9bc3_a1c5d38447da.slice/crio-2146d31ed8ac120d3f364e51f9b245301b13d30a5c6fb5be082414b6261b573a WatchSource:0}: Error finding container 2146d31ed8ac120d3f364e51f9b245301b13d30a5c6fb5be082414b6261b573a: Status 404 returned error can't find the container with id 2146d31ed8ac120d3f364e51f9b245301b13d30a5c6fb5be082414b6261b573a Oct 29 22:33:53.339054 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:53.338817 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" event={"ID":"154474f2-5e45-48f6-9a83-2aa3866e2328","Type":"ContainerStarted","Data":"d118b6dffa627e27ec4a520fcba8b5502841ef009f239adf0165e27cc73c3afe"} Oct 29 22:33:53.342362 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:53.342325 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" event={"ID":"d39741d3-5f7e-4082-9bc3-a1c5d38447da","Type":"ContainerStarted","Data":"2146d31ed8ac120d3f364e51f9b245301b13d30a5c6fb5be082414b6261b573a"} Oct 29 22:33:53.372119 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:53.371989 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:53.487652 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:53.487582 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:54.353811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.353611 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"36cc63c23bd35597f6246754bd6dd0f28a92fdd2c5ce9b68d1e92c717cdd3513"} Oct 29 22:33:54.353811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.353709 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"9d53f09b28393da42e3b5dac73eaf279a52f6d268292aaed9006b9d056497fb0"} Oct 29 22:33:54.353811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.353753 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"50d3648470d8e6f08c5c455ff128612207d6cef0a10d5fbc66b0800386830a07"} Oct 29 22:33:54.462873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.462801 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:54.485640 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.482343 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:33:54.522106 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.520611 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.537319 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.537192 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:33:54.557042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.556855 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.556899 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.556941 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.556973 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557000 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557036 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557070 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557104 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557141 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557173 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557195 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557219 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557247 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54x9m\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557293 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557339 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557363 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557385 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557448 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557467 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.557496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.557490 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.561424 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.561093 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 29 22:33:54.585664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.585309 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 29 22:33:54.585664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.585426 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 29 22:33:54.585664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.585508 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 29 22:33:54.590537 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.590500 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 29 22:33:54.611375 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.611312 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 29 22:33:54.612814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.612791 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 29 22:33:54.613000 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.612981 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zn2tp\"" Oct 29 22:33:54.613082 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613032 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 29 22:33:54.613151 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613081 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 29 22:33:54.613151 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613091 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 29 22:33:54.613260 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613165 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 29 22:33:54.613260 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613186 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 29 22:33:54.613260 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613256 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b1501m0mlj688\"" Oct 29 22:33:54.613424 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613296 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 29 22:33:54.613424 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.613347 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 29 22:33:54.659094 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659028 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659103 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659129 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659165 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659193 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659217 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659252 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659286 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659317 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659353 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659379 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659403 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659427 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659453 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54x9m\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659489 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659548 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659584 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659615 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659659 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.659680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659678 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.660349 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.659926 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.661697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.661389 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.663049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.662900 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.663535 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.663512 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.663797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.663776 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.664321 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.664294 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.665588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.664936 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.665588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.665231 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.665588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.665560 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.666825 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.666805 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.667707 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.667684 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.668809 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.668408 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.668809 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.668776 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.669498 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.669389 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.669915 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.669873 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.670574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.670556 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.671364 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.671321 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.671529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.671507 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.675405 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.675362 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.689728 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.689615 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54x9m\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m\") pod \"prometheus-k8s-0\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:54.720361 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:54.720128 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:33:55.129023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:55.128977 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:33:55.133211 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:33:55.133144 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b438e0_fc10_4ba5_be8d_24f985295624.slice/crio-29cead7bc12b86a79afb29ebe3419efda88fd56eb1db4701ccfcd98a0f46a961 WatchSource:0}: Error finding container 29cead7bc12b86a79afb29ebe3419efda88fd56eb1db4701ccfcd98a0f46a961: Status 404 returned error can't find the container with id 29cead7bc12b86a79afb29ebe3419efda88fd56eb1db4701ccfcd98a0f46a961 Oct 29 22:33:55.152661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:55.152569 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:55.252598 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:55.252508 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:55.363046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:55.363014 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} Oct 29 22:33:55.363406 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:55.363081 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"29cead7bc12b86a79afb29ebe3419efda88fd56eb1db4701ccfcd98a0f46a961"} Oct 29 22:33:56.370632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:56.370599 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" exitCode=0 Oct 29 22:33:56.371062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:56.370666 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} Oct 29 22:33:57.380074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.379799 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" event={"ID":"d39741d3-5f7e-4082-9bc3-a1c5d38447da","Type":"ContainerStarted","Data":"dacdf94bfdfedb346a566ecd532fba28481eb568985fa408daf7832ea205a667"} Oct 29 22:33:57.380620 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.380599 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:57.384658 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.384627 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"67d19ce9a1d87063c2c752206e525ffe4a04486161e6d1a6c874230e84590670"} Oct 29 22:33:57.384756 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.384691 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"d026922c771263d6d5cb9cdb1b7a12e6ff409183b9676903c4719fefb5777ac5"} Oct 29 22:33:57.384756 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.384716 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" event={"ID":"ddd8a66a-5b7f-436e-923f-3a69a9ee546f","Type":"ContainerStarted","Data":"f8d09b49d51af947092ec2c1a8e799bf3400b4ab2a8235dfa4a03884c791dd00"} Oct 29 22:33:57.385005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.384986 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:57.387826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.387789 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" Oct 29 22:33:57.389089 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.389062 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb79dfbcb-ctbr6" event={"ID":"37f89320-c805-4ff5-b530-6bb70cd7169d","Type":"ContainerStarted","Data":"ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848"} Oct 29 22:33:57.394389 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.394351 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c"} Oct 29 22:33:57.394492 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.394442 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52"} Oct 29 22:33:57.394492 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.394487 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438"} Oct 29 22:33:57.394617 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.394527 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd"} Oct 29 22:33:57.394617 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.394568 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c"} Oct 29 22:33:57.394617 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.394607 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerStarted","Data":"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b"} Oct 29 22:33:57.396506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.396480 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" event={"ID":"154474f2-5e45-48f6-9a83-2aa3866e2328","Type":"ContainerStarted","Data":"5df2180c6118de6d4c1abeba62d3fae05af0b439136fedf0fda687640ee2ac30"} Oct 29 22:33:57.424870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.424753 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-blqdr" podStartSLOduration=1.909484266 podStartE2EDuration="5.424737293s" totalImagesPullingTime="3.515253027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:57.423487672 +0000 UTC m=+34.536133270" watchObservedRunningTime="2026-10-29 22:33:57.424737293 +0000 UTC m=+34.537382883" Oct 29 22:33:57.477818 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.477762 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" podStartSLOduration=2.620904213 podStartE2EDuration="8.4777481s" totalImagesPullingTime="5.856843887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:57.477733025 +0000 UTC m=+34.590378635" watchObservedRunningTime="2026-10-29 22:33:57.4777481 +0000 UTC m=+34.590393699" Oct 29 22:33:57.494523 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.494389 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:57.527299 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.527245 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5cb79dfbcb-ctbr6" podStartSLOduration=2.714450518 podStartE2EDuration="9.527229317s" totalImagesPullingTime="6.812778799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:57.525369809 +0000 UTC m=+34.638015398" watchObservedRunningTime="2026-10-29 22:33:57.527229317 +0000 UTC m=+34.639874903" Oct 29 22:33:57.599415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.599362 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:33:57.629716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.629649 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.443597738 podStartE2EDuration="9.629630365s" totalImagesPullingTime="5.168280863s" totalInitContainerRuntime="17.751764ms" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:57.627506615 +0000 UTC m=+34.740152204" watchObservedRunningTime="2026-10-29 22:33:57.629630365 +0000 UTC m=+34.742275955" Oct 29 22:33:57.686160 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:57.685600 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" podStartSLOduration=2.817833871 podStartE2EDuration="6.685585211s" totalImagesPullingTime="3.86775134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:33:57.684619218 +0000 UTC m=+34.797264805" watchObservedRunningTime="2026-10-29 22:33:57.685585211 +0000 UTC m=+34.798230800" Oct 29 22:33:58.409677 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:58.409650 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7986f44ccf-p28v2" Oct 29 22:33:59.336449 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:59.336422 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:59.336649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:59.336473 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:59.343897 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:59.343872 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:33:59.413702 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:33:59.413676 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:34:00.486195 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.486165 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:00.588487 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.587235 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:00.792404 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.792375 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-749797b86b-jk6qf"] Oct 29 22:34:00.816888 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.816040 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749797b86b-jk6qf"] Oct 29 22:34:00.816888 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.816136 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.883420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.883364 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 29 22:34:00.936383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936354 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4lt\" (UniqueName: \"kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936467 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936429 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936467 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936481 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936513 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936534 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936557 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.936602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.936590 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:00.948510 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:00.948458 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:01.038301 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038270 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4lt\" (UniqueName: \"kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038339 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038373 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038387 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038413 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038434 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038458 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.038781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.038489 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.040907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.040065 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.040907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.040361 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.040907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.040741 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.042103 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.042078 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.042308 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.042253 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.042440 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.042420 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.045347 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.045324 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.061426 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.061393 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:01.062076 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.062060 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4lt\" (UniqueName: \"kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt\") pod \"console-749797b86b-jk6qf\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.209421 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.209395 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:01.426275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.426240 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} Oct 29 22:34:01.426426 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.426320 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} Oct 29 22:34:01.426426 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.426360 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} Oct 29 22:34:01.426426 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.426398 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} Oct 29 22:34:01.426522 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.426433 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} Oct 29 22:34:01.426522 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.426466 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerStarted","Data":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} Oct 29 22:34:01.503253 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.503175 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.479089928 podStartE2EDuration="7.50315063s" totalImagesPullingTime="3.999262915s" totalInitContainerRuntime="24.797787ms" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:34:01.501099304 +0000 UTC m=+38.613744895" watchObservedRunningTime="2026-10-29 22:34:01.50315063 +0000 UTC m=+38.615796225" Oct 29 22:34:01.565046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:01.565016 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749797b86b-jk6qf"] Oct 29 22:34:02.239830 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.239806 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68487bc74d-jbpms" Oct 29 22:34:02.374635 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.374599 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fdff46b97-drfgt"] Oct 29 22:34:02.447160 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.447128 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749797b86b-jk6qf" event={"ID":"29c8a0f7-9903-4935-9a71-d0ce49c0996f","Type":"ContainerStarted","Data":"24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf"} Oct 29 22:34:02.447320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.447214 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749797b86b-jk6qf" event={"ID":"29c8a0f7-9903-4935-9a71-d0ce49c0996f","Type":"ContainerStarted","Data":"09d51a6b2f5fa4c010db06c38eb9e316cf4ea985a1305683fe940817d2947070"} Oct 29 22:34:02.475873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.475355 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:02.575436 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.575397 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:02.679862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:02.676618 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:03.466358 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:03.466281 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-r7wrt" event={"ID":"3897b32b-ba8f-41b7-bc91-db7901bb8335","Type":"ContainerStarted","Data":"b1d577666cee51da12ea39e50aaf779f4431640ba2f2dd75aca13baabe0447e9"} Oct 29 22:34:03.466636 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:03.466618 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:34:03.483252 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:03.483224 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-r7wrt" Oct 29 22:34:03.545870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:03.541675 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-749797b86b-jk6qf" podStartSLOduration=3.5416538319999997 podStartE2EDuration="3.541653832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:34:02.493660008 +0000 UTC m=+39.606305596" watchObservedRunningTime="2026-10-29 22:34:03.541653832 +0000 UTC m=+40.654299422" Oct 29 22:34:03.591399 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:03.591337 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-r7wrt" podStartSLOduration=2.306457821 podStartE2EDuration="23.59131826s" totalImagesPullingTime="21.284860439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:34:03.542755545 +0000 UTC m=+40.655401136" watchObservedRunningTime="2026-10-29 22:34:03.59131826 +0000 UTC m=+40.703963849" Oct 29 22:34:04.722920 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:04.720641 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:34:05.168963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.168735 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:05.175303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.175260 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerName="registry" containerID="cri-o://595c3dcdf1ace8a094b81aa81d4dc749fdea78d369fa7e65ce62978d14234689" gracePeriod=30 Oct 29 22:34:05.260383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.260354 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:05.484234 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.484197 5030 generic.go:356] "Generic (PLEG): container finished" podID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerID="595c3dcdf1ace8a094b81aa81d4dc749fdea78d369fa7e65ce62978d14234689" exitCode=0 Oct 29 22:34:05.484797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.484377 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" event={"ID":"eea9c729-9323-4535-bb3b-f5e67bb28b29","Type":"ContainerDied","Data":"595c3dcdf1ace8a094b81aa81d4dc749fdea78d369fa7e65ce62978d14234689"} Oct 29 22:34:05.584048 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.584018 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:34:05.725383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725343 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725387 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725436 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725476 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725588 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725669 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725719 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.725859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.725745 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") pod \"eea9c729-9323-4535-bb3b-f5e67bb28b29\" (UID: \"eea9c729-9323-4535-bb3b-f5e67bb28b29\") " Oct 29 22:34:05.727398 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.726871 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:05.727581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.727022 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:05.732755 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.732681 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "kube-api-access-8nfrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:05.734032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.734005 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:05.734952 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.734919 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:34:05.736264 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.736229 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:34:05.736364 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.736332 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:34:05.737073 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.737033 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls" pod "eea9c729-9323-4535-bb3b-f5e67bb28b29" (UID: "eea9c729-9323-4535-bb3b-f5e67bb28b29"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827381 5030 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-certificates\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827417 5030 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-image-registry-private-configuration\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827434 5030 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eea9c729-9323-4535-bb3b-f5e67bb28b29-ca-trust-extracted\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827451 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nfrs\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-kube-api-access-8nfrs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827467 5030 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eea9c729-9323-4535-bb3b-f5e67bb28b29-trusted-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827481 5030 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eea9c729-9323-4535-bb3b-f5e67bb28b29-installation-pull-secrets\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827495 5030 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-registry-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.830905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:05.827509 5030 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eea9c729-9323-4535-bb3b-f5e67bb28b29-bound-sa-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:05.835468 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:34:05.835433 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:34:06.492205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:06.492172 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" event={"ID":"eea9c729-9323-4535-bb3b-f5e67bb28b29","Type":"ContainerDied","Data":"8aa20cb35c8de461b04f7734a78cabd69f02f446d602fb10ae3477d330b00769"} Oct 29 22:34:06.492382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:06.492235 5030 scope.go:122] "RemoveContainer" containerID="595c3dcdf1ace8a094b81aa81d4dc749fdea78d369fa7e65ce62978d14234689" Oct 29 22:34:06.492448 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:06.492383 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bf65fd8-phdn2" Oct 29 22:34:06.570991 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:06.570955 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66bf65fd8-phdn2"] Oct 29 22:34:06.585546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:06.585514 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66bf65fd8-phdn2"] Oct 29 22:34:07.047096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:07.047038 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" path="/var/lib/kubelet/pods/eea9c729-9323-4535-bb3b-f5e67bb28b29/volumes" Oct 29 22:34:11.210013 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.209984 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:11.210601 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.210050 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:11.214440 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.214420 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:11.330068 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.330038 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:11.429882 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.429817 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:11.539111 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.539086 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:34:11.611973 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.611947 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb79dfbcb-ctbr6"] Oct 29 22:34:11.808073 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.808002 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-s72v6-5hwwc"] Oct 29 22:34:11.808222 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:11.808187 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" podUID="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" containerName="insights-gathering" containerID="cri-o://9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc" gracePeriod=30 Oct 29 22:34:12.149687 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.149665 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:34:12.279500 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.279459 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:12.282358 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.282329 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:34:12.282493 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.282397 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:34:12.302501 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.302476 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") pod \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " Oct 29 22:34:12.302616 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.302543 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") pod \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " Oct 29 22:34:12.302691 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.302676 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") pod \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " Oct 29 22:34:12.302736 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.302701 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") pod \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\" (UID: \"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7\") " Oct 29 22:34:12.302857 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.302821 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle" pod "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" (UID: "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:12.303060 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.303038 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path" pod "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" (UID: "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:34:12.303126 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.303096 5030 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-service-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:12.303126 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.303100 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle" pod "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" (UID: "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:12.304990 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.304971 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm" pod "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" (UID: "56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7"). InnerVolumeSpecName "kube-api-access-5qpwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:12.392244 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.392190 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:12.403864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.403818 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qpwm\" (UniqueName: \"kubernetes.io/projected/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-kube-api-access-5qpwm\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:12.403864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.403861 5030 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-trusted-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:12.403993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.403876 5030 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7-archives-path\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:12.493385 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.493352 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:12.541180 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.541141 5030 generic.go:356] "Generic (PLEG): container finished" podID="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" containerID="9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc" exitCode=2 Oct 29 22:34:12.541336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.541190 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" Oct 29 22:34:12.541336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.541197 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" event={"ID":"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7","Type":"ContainerDied","Data":"9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc"} Oct 29 22:34:12.541336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.541249 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-s72v6-5hwwc" event={"ID":"56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7","Type":"ContainerDied","Data":"e6255efd3425fc3033ff13486257834c17b7233c386908352519f2617a7831b9"} Oct 29 22:34:12.541336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.541289 5030 scope.go:122] "RemoveContainer" containerID="9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc" Oct 29 22:34:12.550884 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.550866 5030 scope.go:122] "RemoveContainer" containerID="9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc" Oct 29 22:34:12.551132 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:34:12.551113 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc\": container with ID starting with 9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc not found: ID does not exist" containerID="9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc" Oct 29 22:34:12.551204 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.551140 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc"} err="failed to get container status \"9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc\": rpc error: code = NotFound desc = could not find container \"9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc\": container with ID starting with 9581297f1efdadd7c38f27e7cd00e89dbe29df25186fad8f34fb8bfed308a7cc not found: ID does not exist" Oct 29 22:34:12.596150 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.596121 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:12.597070 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.597052 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-s72v6-5hwwc"] Oct 29 22:34:12.610526 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:12.610505 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-s72v6-5hwwc"] Oct 29 22:34:13.043908 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:13.043876 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" path="/var/lib/kubelet/pods/56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7/volumes" Oct 29 22:34:13.324527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:13.324440 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:13.426114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:13.426074 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:27.403811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.403750 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerName="registry" containerID="cri-o://2dcfa06cf84302aa68d234328a29b58428ac9250526dbcc8e5cc781e566ebe7f" gracePeriod=30 Oct 29 22:34:27.550359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.550324 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:27.618488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.618451 5030 generic.go:356] "Generic (PLEG): container finished" podID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerID="2dcfa06cf84302aa68d234328a29b58428ac9250526dbcc8e5cc781e566ebe7f" exitCode=0 Oct 29 22:34:27.618633 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.618529 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" event={"ID":"05dc76a9-2747-4bd0-907d-a9b2caa3b25d","Type":"ContainerDied","Data":"2dcfa06cf84302aa68d234328a29b58428ac9250526dbcc8e5cc781e566ebe7f"} Oct 29 22:34:27.667039 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.667005 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:27.690340 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.690317 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:34:27.739318 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739296 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739332 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739383 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739407 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739450 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739477 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739523 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.739708 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.739614 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") pod \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\" (UID: \"05dc76a9-2747-4bd0-907d-a9b2caa3b25d\") " Oct 29 22:34:27.740200 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.740136 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:27.740661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.740455 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:27.741694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.741674 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:27.742282 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.742241 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:34:27.742444 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.742417 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:34:27.742444 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.742437 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:27.742589 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.742517 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "kube-api-access-vmsj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:27.747538 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.747519 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted" pod "05dc76a9-2747-4bd0-907d-a9b2caa3b25d" (UID: "05dc76a9-2747-4bd0-907d-a9b2caa3b25d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:34:27.841257 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841231 5030 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-trusted-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841257 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841255 5030 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-image-registry-private-configuration\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841266 5030 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-certificates\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841276 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmsj6\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-kube-api-access-vmsj6\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841284 5030 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-registry-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841292 5030 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-ca-trust-extracted\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841300 5030 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-bound-sa-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:27.841439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:27.841309 5030 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dc76a9-2747-4bd0-907d-a9b2caa3b25d-installation-pull-secrets\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:28.624334 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:28.624309 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" Oct 29 22:34:28.624927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:28.624345 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fdff46b97-drfgt" event={"ID":"05dc76a9-2747-4bd0-907d-a9b2caa3b25d","Type":"ContainerDied","Data":"86d2e23d329dab9463c2e46921ded8cf379bb7e5ef5c4e931ac4912d5e0044fb"} Oct 29 22:34:28.624927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:28.624399 5030 scope.go:122] "RemoveContainer" containerID="2dcfa06cf84302aa68d234328a29b58428ac9250526dbcc8e5cc781e566ebe7f" Oct 29 22:34:28.694481 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:28.694453 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fdff46b97-drfgt"] Oct 29 22:34:28.708389 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:28.708363 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fdff46b97-drfgt"] Oct 29 22:34:29.044233 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:29.044206 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" path="/var/lib/kubelet/pods/05dc76a9-2747-4bd0-907d-a9b2caa3b25d/volumes" Oct 29 22:34:32.288648 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:32.288613 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:34:32.292266 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:32.292247 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-77dd54f899-2qj6z" Oct 29 22:34:34.948763 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:34.948728 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:35.032857 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:35.032788 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:35.793553 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:34:35.793518 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:34:36.637726 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:36.637694 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5cb79dfbcb-ctbr6" podUID="37f89320-c805-4ff5-b530-6bb70cd7169d" containerName="console" containerID="cri-o://ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848" gracePeriod=15 Oct 29 22:34:36.948489 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:36.948464 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:34:37.026425 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026401 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp\" (UniqueName: \"kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.026581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026457 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.026581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026532 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.026581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026557 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf\" (UniqueName: \"kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.026581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026579 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.026777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026637 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.026777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026712 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:34:37.026942 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026788 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config\") pod \"37f89320-c805-4ff5-b530-6bb70cd7169d\" (UID: \"37f89320-c805-4ff5-b530-6bb70cd7169d\") " Oct 29 22:34:37.027001 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.026971 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:37.027055 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.027029 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:37.027305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.027279 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:37.027622 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.027291 5030 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37f89320-c805-4ff5-b530-6bb70cd7169d-tmp\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.027725 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.027634 5030 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-oauth-serving-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.027725 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.027651 5030 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.029159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.029132 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:34:37.029159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.029141 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:34:37.029286 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.029194 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf" pod "37f89320-c805-4ff5-b530-6bb70cd7169d" (UID: "37f89320-c805-4ff5-b530-6bb70cd7169d"). InnerVolumeSpecName "kube-api-access-m4cqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:37.129440 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.129418 5030 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-oauth-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.129573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.129442 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4cqf\" (UniqueName: \"kubernetes.io/projected/37f89320-c805-4ff5-b530-6bb70cd7169d-kube-api-access-m4cqf\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.129573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.129457 5030 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f89320-c805-4ff5-b530-6bb70cd7169d-console-serving-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.129573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.129470 5030 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f89320-c805-4ff5-b530-6bb70cd7169d-console-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:37.674376 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.674345 5030 generic.go:356] "Generic (PLEG): container finished" podID="37f89320-c805-4ff5-b530-6bb70cd7169d" containerID="ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848" exitCode=0 Oct 29 22:34:37.674860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.674393 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb79dfbcb-ctbr6" Oct 29 22:34:37.674860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.674441 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb79dfbcb-ctbr6" event={"ID":"37f89320-c805-4ff5-b530-6bb70cd7169d","Type":"ContainerDied","Data":"ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848"} Oct 29 22:34:37.674860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.674502 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb79dfbcb-ctbr6" event={"ID":"37f89320-c805-4ff5-b530-6bb70cd7169d","Type":"ContainerDied","Data":"879b5961684ff8f26633e801d6292f5fcbe33ffc9956b22365e769eb036c836d"} Oct 29 22:34:37.674860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.674534 5030 scope.go:122] "RemoveContainer" containerID="ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848" Oct 29 22:34:37.683474 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.683451 5030 scope.go:122] "RemoveContainer" containerID="ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848" Oct 29 22:34:37.683752 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:34:37.683734 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848\": container with ID starting with ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848 not found: ID does not exist" containerID="ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848" Oct 29 22:34:37.683801 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.683761 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848"} err="failed to get container status \"ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848\": rpc error: code = NotFound desc = could not find container \"ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848\": container with ID starting with ccc32d1aff52e0cc12aa53669d464481e9d0bb7018872b51611e63fb127cc848 not found: ID does not exist" Oct 29 22:34:37.731060 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.731037 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb79dfbcb-ctbr6"] Oct 29 22:34:37.744700 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:37.744667 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb79dfbcb-ctbr6"] Oct 29 22:34:38.681010 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:38.680924 5030 generic.go:356] "Generic (PLEG): container finished" podID="ed3a898c-87c6-409d-a70f-f01e8e15d787" containerID="f78bf227f7d6997b403dba59695a13e0f442e113b14c9908a89bd45fd6b617bf" exitCode=0 Oct 29 22:34:38.681010 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:38.681000 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" event={"ID":"ed3a898c-87c6-409d-a70f-f01e8e15d787","Type":"ContainerDied","Data":"f78bf227f7d6997b403dba59695a13e0f442e113b14c9908a89bd45fd6b617bf"} Oct 29 22:34:38.681531 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:38.681056 5030 scope.go:122] "RemoveContainer" containerID="f9d3e2835828fd3d57f02bdc7028af74cff9a6e1b251467c3d0e3f75c0f364d7" Oct 29 22:34:38.681531 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:38.681394 5030 scope.go:122] "RemoveContainer" containerID="f78bf227f7d6997b403dba59695a13e0f442e113b14c9908a89bd45fd6b617bf" Oct 29 22:34:38.681669 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:34:38.681650 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"insights-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=insights-operator pod=insights-operator-7c4986d449-8p2vc_openshift-insights(ed3a898c-87c6-409d-a70f-f01e8e15d787)\"" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" podUID="ed3a898c-87c6-409d-a70f-f01e8e15d787" Oct 29 22:34:39.043695 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:39.043662 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37f89320-c805-4ff5-b530-6bb70cd7169d" path="/var/lib/kubelet/pods/37f89320-c805-4ff5-b530-6bb70cd7169d/volumes" Oct 29 22:34:49.037993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:49.037960 5030 scope.go:122] "RemoveContainer" containerID="f78bf227f7d6997b403dba59695a13e0f442e113b14c9908a89bd45fd6b617bf" Oct 29 22:34:49.736592 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:49.736561 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-8p2vc" event={"ID":"ed3a898c-87c6-409d-a70f-f01e8e15d787","Type":"ContainerStarted","Data":"7df422e25586389a8759ea327083ab7a2e865d02cefe14950531444867a75d1d"} Oct 29 22:34:49.841792 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:49.841760 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-hj5vq-jp65r"] Oct 29 22:34:49.842007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:49.841975 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" podUID="9fb7b98b-4661-47a2-8c64-97e7efc2d381" containerName="insights-gathering" containerID="cri-o://5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772" gracePeriod=30 Oct 29 22:34:50.108720 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.108694 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:34:50.244653 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.244624 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx\" (UniqueName: \"kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx\") pod \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " Oct 29 22:34:50.244859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.244665 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle\") pod \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " Oct 29 22:34:50.244859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.244702 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path\") pod \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " Oct 29 22:34:50.244948 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.244832 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle\") pod \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\" (UID: \"9fb7b98b-4661-47a2-8c64-97e7efc2d381\") " Oct 29 22:34:50.245117 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.245083 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path" pod "9fb7b98b-4661-47a2-8c64-97e7efc2d381" (UID: "9fb7b98b-4661-47a2-8c64-97e7efc2d381"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:34:50.245176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.245141 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle" pod "9fb7b98b-4661-47a2-8c64-97e7efc2d381" (UID: "9fb7b98b-4661-47a2-8c64-97e7efc2d381"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:50.245176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.245136 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle" pod "9fb7b98b-4661-47a2-8c64-97e7efc2d381" (UID: "9fb7b98b-4661-47a2-8c64-97e7efc2d381"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:34:50.245241 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.245227 5030 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-trusted-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:50.245274 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.245241 5030 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9fb7b98b-4661-47a2-8c64-97e7efc2d381-archives-path\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:50.245274 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.245249 5030 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fb7b98b-4661-47a2-8c64-97e7efc2d381-service-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:50.246865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.246825 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx" pod "9fb7b98b-4661-47a2-8c64-97e7efc2d381" (UID: "9fb7b98b-4661-47a2-8c64-97e7efc2d381"). InnerVolumeSpecName "kube-api-access-bp5nx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:34:50.330107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.330052 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:34:50.346239 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.346206 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp5nx\" (UniqueName: \"kubernetes.io/projected/9fb7b98b-4661-47a2-8c64-97e7efc2d381-kube-api-access-bp5nx\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:34:50.346491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.346473 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:34:50.632135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.632070 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:50.741958 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.741923 5030 generic.go:356] "Generic (PLEG): container finished" podID="9fb7b98b-4661-47a2-8c64-97e7efc2d381" containerID="5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772" exitCode=2 Oct 29 22:34:50.742129 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.741983 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" Oct 29 22:34:50.742129 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.742000 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" event={"ID":"9fb7b98b-4661-47a2-8c64-97e7efc2d381","Type":"ContainerDied","Data":"5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772"} Oct 29 22:34:50.742129 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.742070 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hj5vq-jp65r" event={"ID":"9fb7b98b-4661-47a2-8c64-97e7efc2d381","Type":"ContainerDied","Data":"eae3bbb64cbb55f368d2ed4aedb47589a7df78b121430de50ba8076d2d9a7a50"} Oct 29 22:34:50.742129 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.742091 5030 scope.go:122] "RemoveContainer" containerID="5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772" Oct 29 22:34:50.753749 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.753729 5030 scope.go:122] "RemoveContainer" containerID="5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772" Oct 29 22:34:50.754075 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:34:50.754056 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772\": container with ID starting with 5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772 not found: ID does not exist" containerID="5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772" Oct 29 22:34:50.754180 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.754085 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772"} err="failed to get container status \"5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772\": rpc error: code = NotFound desc = could not find container \"5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772\": container with ID starting with 5a7662b0d05721ee4b41ca8f79573e387f0575acb537e037b2e3fb6776fba772 not found: ID does not exist" Oct 29 22:34:50.761731 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.761714 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:34:50.797648 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.797618 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-hj5vq-jp65r"] Oct 29 22:34:50.801736 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:50.801718 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-hj5vq-jp65r"] Oct 29 22:34:51.043907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:51.043878 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fb7b98b-4661-47a2-8c64-97e7efc2d381" path="/var/lib/kubelet/pods/9fb7b98b-4661-47a2-8c64-97e7efc2d381/volumes" Oct 29 22:34:57.777927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:57.777894 5030 generic.go:356] "Generic (PLEG): container finished" podID="d8b45469-5973-49bc-b8c2-8ec0dce0e242" containerID="814f49e837d5fc2a5c057f4343d4d6c092c26b5c13039857219074b4ca657c59" exitCode=0 Oct 29 22:34:57.778320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:57.777968 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" event={"ID":"d8b45469-5973-49bc-b8c2-8ec0dce0e242","Type":"ContainerDied","Data":"814f49e837d5fc2a5c057f4343d4d6c092c26b5c13039857219074b4ca657c59"} Oct 29 22:34:57.778362 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:57.778329 5030 scope.go:122] "RemoveContainer" containerID="814f49e837d5fc2a5c057f4343d4d6c092c26b5c13039857219074b4ca657c59" Oct 29 22:34:57.849231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:57.849183 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:58.004007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:58.003976 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:34:58.784797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:58.784759 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-hhk2d" event={"ID":"d8b45469-5973-49bc-b8c2-8ec0dce0e242","Type":"ContainerStarted","Data":"957cc5d5bf592fd330de1e91b0a810b1ff1616184910cd6c40d768422c46af90"} Oct 29 22:34:59.192407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192336 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:34:59.192684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192652 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="alertmanager" containerID="cri-o://ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b" gracePeriod=120 Oct 29 22:34:59.192937 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192687 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-metric" containerID="cri-o://794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52" gracePeriod=120 Oct 29 22:34:59.193051 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192715 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="prom-label-proxy" containerID="cri-o://9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c" gracePeriod=120 Oct 29 22:34:59.193113 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192726 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-web" containerID="cri-o://8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd" gracePeriod=120 Oct 29 22:34:59.193167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192733 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="config-reloader" containerID="cri-o://1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c" gracePeriod=120 Oct 29 22:34:59.193225 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.192761 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy" containerID="cri-o://f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438" gracePeriod=120 Oct 29 22:34:59.793203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793160 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c" exitCode=0 Oct 29 22:34:59.793203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793196 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438" exitCode=0 Oct 29 22:34:59.793203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793206 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c" exitCode=0 Oct 29 22:34:59.793698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793215 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b" exitCode=0 Oct 29 22:34:59.793698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793190 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c"} Oct 29 22:34:59.793698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793317 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438"} Oct 29 22:34:59.793698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793370 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c"} Oct 29 22:34:59.793698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:34:59.793416 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b"} Oct 29 22:35:00.457136 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.457114 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:00.641639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641564 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.641639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641593 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.641639 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641625 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.641934 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641674 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.641934 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641902 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641951 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.641999 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:35:00.642049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642029 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642066 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642109 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642137 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642173 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642421 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642231 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.642421 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642263 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle\") pod \"9d90f791-07a7-47a8-8ba1-07b424c37404\" (UID: \"9d90f791-07a7-47a8-8ba1-07b424c37404\") " Oct 29 22:35:00.643020 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642791 5030 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-main-db\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.643020 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.642956 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:00.643479 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.643452 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:00.645087 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.645059 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:35:00.645260 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.645229 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.645738 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.645709 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.646285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.646045 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.646285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.646170 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:35:00.646455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.646300 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "kube-api-access-g2cbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:35:00.646600 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.646583 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.646890 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.646873 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.651968 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.651857 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.659976 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.659955 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config" pod "9d90f791-07a7-47a8-8ba1-07b424c37404" (UID: "9d90f791-07a7-47a8-8ba1-07b424c37404"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:00.743499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743474 5030 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-tls-assets\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743494 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-metrics-client-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743503 5030 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d90f791-07a7-47a8-8ba1-07b424c37404-config-out\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743511 5030 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743521 5030 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743530 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2cbr\" (UniqueName: \"kubernetes.io/projected/9d90f791-07a7-47a8-8ba1-07b424c37404-kube-api-access-g2cbr\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743539 5030 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d90f791-07a7-47a8-8ba1-07b424c37404-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743548 5030 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-web-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743556 5030 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-cluster-tls-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743565 5030 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743573 5030 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-config-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.743613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.743581 5030 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d90f791-07a7-47a8-8ba1-07b424c37404-secret-alertmanager-main-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:00.801525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801498 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52" exitCode=0 Oct 29 22:35:00.801525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801521 5030 generic.go:356] "Generic (PLEG): container finished" podID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerID="8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd" exitCode=0 Oct 29 22:35:00.801927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801591 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:00.801927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801583 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52"} Oct 29 22:35:00.801927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801720 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd"} Oct 29 22:35:00.801927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801751 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d90f791-07a7-47a8-8ba1-07b424c37404","Type":"ContainerDied","Data":"ded61dc51bb5638feaa7e38bbba6f77a3e7259a7c8f4581e4af5d6c8a3d29fb3"} Oct 29 22:35:00.801927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.801786 5030 scope.go:122] "RemoveContainer" containerID="9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c" Oct 29 22:35:00.814596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.810510 5030 scope.go:122] "RemoveContainer" containerID="794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52" Oct 29 22:35:00.826064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.826026 5030 scope.go:122] "RemoveContainer" containerID="f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438" Oct 29 22:35:00.833160 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.833139 5030 scope.go:122] "RemoveContainer" containerID="8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd" Oct 29 22:35:00.840279 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.840260 5030 scope.go:122] "RemoveContainer" containerID="1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c" Oct 29 22:35:00.847552 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.847536 5030 scope.go:122] "RemoveContainer" containerID="ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b" Oct 29 22:35:00.854825 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.854810 5030 scope.go:122] "RemoveContainer" containerID="cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568" Oct 29 22:35:00.861796 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.861773 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:35:00.862292 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.862279 5030 scope.go:122] "RemoveContainer" containerID="9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c" Oct 29 22:35:00.862558 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.862540 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c\": container with ID starting with 9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c not found: ID does not exist" containerID="9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c" Oct 29 22:35:00.862634 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.862570 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c"} err="failed to get container status \"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c\": rpc error: code = NotFound desc = could not find container \"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c\": container with ID starting with 9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c not found: ID does not exist" Oct 29 22:35:00.862634 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.862593 5030 scope.go:122] "RemoveContainer" containerID="794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52" Oct 29 22:35:00.862831 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.862815 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52\": container with ID starting with 794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52 not found: ID does not exist" containerID="794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52" Oct 29 22:35:00.863020 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.862898 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52"} err="failed to get container status \"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52\": rpc error: code = NotFound desc = could not find container \"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52\": container with ID starting with 794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52 not found: ID does not exist" Oct 29 22:35:00.863020 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.862920 5030 scope.go:122] "RemoveContainer" containerID="f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438" Oct 29 22:35:00.863183 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.863164 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438\": container with ID starting with f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438 not found: ID does not exist" containerID="f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438" Oct 29 22:35:00.863243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863189 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438"} err="failed to get container status \"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438\": rpc error: code = NotFound desc = could not find container \"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438\": container with ID starting with f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438 not found: ID does not exist" Oct 29 22:35:00.863243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863203 5030 scope.go:122] "RemoveContainer" containerID="8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd" Oct 29 22:35:00.863431 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.863407 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd\": container with ID starting with 8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd not found: ID does not exist" containerID="8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd" Oct 29 22:35:00.863517 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863435 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd"} err="failed to get container status \"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd\": rpc error: code = NotFound desc = could not find container \"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd\": container with ID starting with 8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd not found: ID does not exist" Oct 29 22:35:00.863517 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863454 5030 scope.go:122] "RemoveContainer" containerID="1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c" Oct 29 22:35:00.863669 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.863649 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c\": container with ID starting with 1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c not found: ID does not exist" containerID="1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c" Oct 29 22:35:00.863719 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863671 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c"} err="failed to get container status \"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c\": rpc error: code = NotFound desc = could not find container \"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c\": container with ID starting with 1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c not found: ID does not exist" Oct 29 22:35:00.863719 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863684 5030 scope.go:122] "RemoveContainer" containerID="ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b" Oct 29 22:35:00.863902 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.863885 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b\": container with ID starting with ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b not found: ID does not exist" containerID="ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b" Oct 29 22:35:00.863957 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863909 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b"} err="failed to get container status \"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b\": rpc error: code = NotFound desc = could not find container \"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b\": container with ID starting with ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b not found: ID does not exist" Oct 29 22:35:00.863957 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.863924 5030 scope.go:122] "RemoveContainer" containerID="cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568" Oct 29 22:35:00.864163 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:00.864148 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568\": container with ID starting with cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568 not found: ID does not exist" containerID="cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568" Oct 29 22:35:00.864199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864166 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568"} err="failed to get container status \"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568\": rpc error: code = NotFound desc = could not find container \"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568\": container with ID starting with cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568 not found: ID does not exist" Oct 29 22:35:00.864199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864177 5030 scope.go:122] "RemoveContainer" containerID="9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c" Oct 29 22:35:00.864408 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864366 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c"} err="failed to get container status \"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c\": rpc error: code = NotFound desc = could not find container \"9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c\": container with ID starting with 9ce4638e0d6593a7979d1a27d4c0bc5511c7dd703419077cc0e9691a5231899c not found: ID does not exist" Oct 29 22:35:00.864408 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864383 5030 scope.go:122] "RemoveContainer" containerID="794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52" Oct 29 22:35:00.864620 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864602 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52"} err="failed to get container status \"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52\": rpc error: code = NotFound desc = could not find container \"794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52\": container with ID starting with 794927018e131dfa19e6cd7ef76ef4c8e9e2e4680571debd569b51e187a09f52 not found: ID does not exist" Oct 29 22:35:00.864665 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864622 5030 scope.go:122] "RemoveContainer" containerID="f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438" Oct 29 22:35:00.864878 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864827 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438"} err="failed to get container status \"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438\": rpc error: code = NotFound desc = could not find container \"f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438\": container with ID starting with f66658b8a5a1431a73b5ee7545056feec4cedbb61bfb97369c5d7be05002d438 not found: ID does not exist" Oct 29 22:35:00.864878 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.864857 5030 scope.go:122] "RemoveContainer" containerID="8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd" Oct 29 22:35:00.865089 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865072 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd"} err="failed to get container status \"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd\": rpc error: code = NotFound desc = could not find container \"8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd\": container with ID starting with 8b7c20099735dd267d38a312c1761472aaff83b9540467f65b05c4a33b0ef1dd not found: ID does not exist" Oct 29 22:35:00.865149 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865089 5030 scope.go:122] "RemoveContainer" containerID="1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c" Oct 29 22:35:00.865315 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865299 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c"} err="failed to get container status \"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c\": rpc error: code = NotFound desc = could not find container \"1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c\": container with ID starting with 1241ce60d073cc1d993db3baba8fd9087672632638065b709b4031334fd6b34c not found: ID does not exist" Oct 29 22:35:00.865378 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865315 5030 scope.go:122] "RemoveContainer" containerID="ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b" Oct 29 22:35:00.865532 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865507 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b"} err="failed to get container status \"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b\": rpc error: code = NotFound desc = could not find container \"ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b\": container with ID starting with ab77b80150f1d4137bcefdc6c484f7c9202a98109792f63ba17240c7c56ed06b not found: ID does not exist" Oct 29 22:35:00.865532 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865525 5030 scope.go:122] "RemoveContainer" containerID="cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568" Oct 29 22:35:00.865720 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.865705 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568"} err="failed to get container status \"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568\": rpc error: code = NotFound desc = could not find container \"cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568\": container with ID starting with cfe49e60f492edbd8ed25369da17135b68d04bfc43fdd1d5c83869ff0fd83568 not found: ID does not exist" Oct 29 22:35:00.876259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.876233 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:35:00.890189 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890167 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:35:00.890621 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890604 5030 cpu_manager.go:414] "containerMap: removing container" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerName="registry" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890623 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerName="registry" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890640 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9fb7b98b-4661-47a2-8c64-97e7efc2d381" containerName="insights-gathering" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890648 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb7b98b-4661-47a2-8c64-97e7efc2d381" containerName="insights-gathering" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890661 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-web" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890669 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-web" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890685 5030 cpu_manager.go:414] "containerMap: removing container" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerName="registry" Oct 29 22:35:00.890693 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890692 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerName="registry" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890710 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="init-config-reloader" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890718 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="init-config-reloader" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890737 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="alertmanager" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890744 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="alertmanager" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890764 5030 cpu_manager.go:414] "containerMap: removing container" podUID="37f89320-c805-4ff5-b530-6bb70cd7169d" containerName="console" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890772 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37f89320-c805-4ff5-b530-6bb70cd7169d" containerName="console" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890792 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="config-reloader" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890799 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="config-reloader" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890812 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="prom-label-proxy" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890820 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="prom-label-proxy" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890859 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-metric" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890867 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-metric" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890878 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890884 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890897 5030 cpu_manager.go:414] "containerMap: removing container" podUID="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" containerName="insights-gathering" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.890904 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" containerName="insights-gathering" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891007 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="56290f6c-9bcf-44ca-b44c-aa29ac9cbcc7" containerName="insights-gathering" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891022 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="05dc76a9-2747-4bd0-907d-a9b2caa3b25d" containerName="registry" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891032 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb7b98b-4661-47a2-8c64-97e7efc2d381" containerName="insights-gathering" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891041 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-web" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891051 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="eea9c729-9323-4535-bb3b-f5e67bb28b29" containerName="registry" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891062 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="alertmanager" Oct 29 22:35:00.891062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891073 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="37f89320-c805-4ff5-b530-6bb70cd7169d" containerName="console" Oct 29 22:35:00.892159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891083 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="config-reloader" Oct 29 22:35:00.892159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891093 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="prom-label-proxy" Oct 29 22:35:00.892159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891105 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy-metric" Oct 29 22:35:00.892159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.891113 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" containerName="kube-rbac-proxy" Oct 29 22:35:00.895757 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.895737 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:00.912759 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912735 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 29 22:35:00.912881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912746 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 29 22:35:00.912881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912802 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 29 22:35:00.912881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912811 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 29 22:35:00.912881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912809 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 29 22:35:00.913093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912885 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 29 22:35:00.913093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912907 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 29 22:35:00.913093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.912971 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-54n2z\"" Oct 29 22:35:00.927992 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.927972 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 29 22:35:00.929872 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:00.929853 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:35:01.013377 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.013355 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 29 22:35:01.043564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.043540 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d90f791-07a7-47a8-8ba1-07b424c37404" path="/var/lib/kubelet/pods/9d90f791-07a7-47a8-8ba1-07b424c37404/volumes" Oct 29 22:35:01.046745 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.046711 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ff86b1f-1358-4900-becc-5ab1f332ed9a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.046879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.046749 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ff86b1f-1358-4900-becc-5ab1f332ed9a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.046879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.046770 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.046879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.046814 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.046928 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ff86b1f-1358-4900-becc-5ab1f332ed9a-config-out\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.046966 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047016 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ff86b1f-1358-4900-becc-5ab1f332ed9a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047165 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047107 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047165 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047141 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ff86b1f-1358-4900-becc-5ab1f332ed9a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047165 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047160 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047192 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-web-config\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047228 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.047352 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.047335 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th7mg\" (UniqueName: \"kubernetes.io/projected/1ff86b1f-1358-4900-becc-5ab1f332ed9a-kube-api-access-th7mg\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148029 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ff86b1f-1358-4900-becc-5ab1f332ed9a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148062 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148084 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ff86b1f-1358-4900-becc-5ab1f332ed9a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148124 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148479 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148458 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ff86b1f-1358-4900-becc-5ab1f332ed9a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148600 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148575 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-web-config\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148621 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.148715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148661 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th7mg\" (UniqueName: \"kubernetes.io/projected/1ff86b1f-1358-4900-becc-5ab1f332ed9a-kube-api-access-th7mg\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.149023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148726 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ff86b1f-1358-4900-becc-5ab1f332ed9a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.149023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148767 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ff86b1f-1358-4900-becc-5ab1f332ed9a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.149023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148808 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.149023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148852 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.149023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148916 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ff86b1f-1358-4900-becc-5ab1f332ed9a-config-out\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.149023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.148954 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.151938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151189 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.151938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151376 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ff86b1f-1358-4900-becc-5ab1f332ed9a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.151938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151511 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-web-config\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.151938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151671 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ff86b1f-1358-4900-becc-5ab1f332ed9a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.151938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151691 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.151938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151826 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.152300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.151983 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ff86b1f-1358-4900-becc-5ab1f332ed9a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.152300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.152241 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.152477 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.152459 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.153014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.152994 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ff86b1f-1358-4900-becc-5ab1f332ed9a-config-out\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.153789 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.153771 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ff86b1f-1358-4900-becc-5ab1f332ed9a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.169993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.169976 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th7mg\" (UniqueName: \"kubernetes.io/projected/1ff86b1f-1358-4900-becc-5ab1f332ed9a-kube-api-access-th7mg\") pod \"alertmanager-main-0\" (UID: \"1ff86b1f-1358-4900-becc-5ab1f332ed9a\") " pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.214575 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.214560 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 29 22:35:01.552411 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.552378 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 29 22:35:01.554250 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:35:01.554224 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff86b1f_1358_4900_becc_5ab1f332ed9a.slice/crio-b28d0320c642c3c58e0bc1c1cc7817646210eb85388f24eef104c09d15585a58 WatchSource:0}: Error finding container b28d0320c642c3c58e0bc1c1cc7817646210eb85388f24eef104c09d15585a58: Status 404 returned error can't find the container with id b28d0320c642c3c58e0bc1c1cc7817646210eb85388f24eef104c09d15585a58 Oct 29 22:35:01.809832 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.809755 5030 generic.go:356] "Generic (PLEG): container finished" podID="1ff86b1f-1358-4900-becc-5ab1f332ed9a" containerID="28ca740f0f3e046f289b4c007adecb722c9fea612d79c57dffc78e34be98f3c1" exitCode=0 Oct 29 22:35:01.809832 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.809817 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerDied","Data":"28ca740f0f3e046f289b4c007adecb722c9fea612d79c57dffc78e34be98f3c1"} Oct 29 22:35:01.810324 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:01.809913 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"b28d0320c642c3c58e0bc1c1cc7817646210eb85388f24eef104c09d15585a58"} Oct 29 22:35:02.819463 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.819430 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"a2943265469aec5cf1265cfbb16a8e2227438afd3b26c2a1564f2fcf6f432380"} Oct 29 22:35:02.819959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.819529 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"29a6bcbe3eb7c8ed4ebb165a9e045ff08fd9407733ca03b0c97d74da31b2331a"} Oct 29 22:35:02.819959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.819554 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"c14b7a21ccf67f631a22165882a2055867e4346053c4cd4d9e3716629f72b575"} Oct 29 22:35:02.819959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.819579 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"11d9733ed8c3ac8ff69cd8a7c19e3d491df9f62dc6e6d3a23b3adf3dae21053a"} Oct 29 22:35:02.819959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.819607 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"d93aa1ff8d9ac57fca424fd43b48f7549810bd90cc60bdc95bde395da590f0ef"} Oct 29 22:35:02.819959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.819633 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ff86b1f-1358-4900-becc-5ab1f332ed9a","Type":"ContainerStarted","Data":"ed018e0a8684f6116935af8b445a9503a912f0ea744f60d8dfd0d4aa5fc97c16"} Oct 29 22:35:02.898345 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:02.898293 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8876644479999998 podStartE2EDuration="2.898274102s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.609654ms" isStatefulPod=false podCreationTimestamp="2026-10-29 22:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:35:02.88382162 +0000 UTC m=+99.996467209" watchObservedRunningTime="2026-10-29 22:35:02.898274102 +0000 UTC m=+100.010919693" Oct 29 22:35:03.158640 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.158535 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-58bd5db97b-9zfb8"] Oct 29 22:35:03.161953 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.161930 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165618 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165582 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-serving-certs-ca-bundle\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165748 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165631 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-telemeter-client-tls\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165748 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165735 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165849 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165793 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-federate-client-tls\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165851 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165959 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165948 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-secret-telemeter-client\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.165994 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165969 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh6ck\" (UniqueName: \"kubernetes.io/projected/b6e248fe-a23c-46ed-b15d-6b318912801d-kube-api-access-dh6ck\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.166036 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.165992 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-metrics-client-ca\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.182279 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.182254 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Oct 29 22:35:03.182406 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.182286 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Oct 29 22:35:03.182406 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.182314 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Oct 29 22:35:03.182503 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.182415 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-j4x66\"" Oct 29 22:35:03.182503 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.182488 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Oct 29 22:35:03.198827 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.198804 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58bd5db97b-9zfb8"] Oct 29 22:35:03.265977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.265952 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Oct 29 22:35:03.266481 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266460 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-secret-telemeter-client\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266491 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh6ck\" (UniqueName: \"kubernetes.io/projected/b6e248fe-a23c-46ed-b15d-6b318912801d-kube-api-access-dh6ck\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266518 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-metrics-client-ca\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266552 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-serving-certs-ca-bundle\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266733 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266574 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-telemeter-client-tls\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266733 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266702 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266762 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-federate-client-tls\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.266865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.266790 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.267443 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.267419 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-serving-certs-ca-bundle\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.267589 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.267534 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-metrics-client-ca\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.269271 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.269248 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-telemeter-client-tls\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.269423 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.269406 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-federate-client-tls\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.269616 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.269601 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-secret-telemeter-client\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.269657 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.269613 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6e248fe-a23c-46ed-b15d-6b318912801d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.281079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.281058 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Oct 29 22:35:03.287489 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.287468 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e248fe-a23c-46ed-b15d-6b318912801d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.288978 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.288958 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh6ck\" (UniqueName: \"kubernetes.io/projected/b6e248fe-a23c-46ed-b15d-6b318912801d-kube-api-access-dh6ck\") pod \"telemeter-client-58bd5db97b-9zfb8\" (UID: \"b6e248fe-a23c-46ed-b15d-6b318912801d\") " pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.476464 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.476431 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" Oct 29 22:35:03.813198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.813129 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58bd5db97b-9zfb8"] Oct 29 22:35:03.830879 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:35:03.830564 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e248fe_a23c_46ed_b15d_6b318912801d.slice/crio-c3b7fbfd8281b5af7a0c19ee809b84d4845c15884b23bbc3e91be5f8bda4ba36 WatchSource:0}: Error finding container c3b7fbfd8281b5af7a0c19ee809b84d4845c15884b23bbc3e91be5f8bda4ba36: Status 404 returned error can't find the container with id c3b7fbfd8281b5af7a0c19ee809b84d4845c15884b23bbc3e91be5f8bda4ba36 Oct 29 22:35:03.843958 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.843787 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:35:03.844328 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.844168 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="prometheus" containerID="cri-o://85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" gracePeriod=600 Oct 29 22:35:03.844328 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.844196 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy" containerID="cri-o://0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" gracePeriod=600 Oct 29 22:35:03.844328 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.844262 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="config-reloader" containerID="cri-o://4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" gracePeriod=600 Oct 29 22:35:03.844516 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.844340 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" gracePeriod=600 Oct 29 22:35:03.844516 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.844210 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-web" containerID="cri-o://48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" gracePeriod=600 Oct 29 22:35:03.844516 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:03.844347 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="thanos-sidecar" containerID="cri-o://cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" gracePeriod=600 Oct 29 22:35:04.108961 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.108932 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.173529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173497 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.173683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173624 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.173683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173659 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.173817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173692 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.173817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173727 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.173817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173759 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.173817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173785 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173823 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173822 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173871 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173913 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173968 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.173996 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174037 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174041 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.174488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174081 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174140 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.174488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174481 5030 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.174646 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174500 5030 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-metrics-client-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.174701 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.174655 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.175096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.175072 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.175415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.175387 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:35:04.175788 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.175766 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.176069 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.176031 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.177249 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.177196 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.177416 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.177384 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:35:04.178300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.178270 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.178501 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.178476 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:35:04.178804 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.178776 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.178804 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.178793 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.179041 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.179014 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.195200 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.195175 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.274921 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.274887 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.275074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.274939 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.275074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.274980 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.275074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275009 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.275074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275050 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m\") pod \"46b438e0-fc10-4ba5-be8d-24f985295624\" (UID: \"46b438e0-fc10-4ba5-be8d-24f985295624\") " Oct 29 22:35:04.275523 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275496 5030 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-config-out\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275527 5030 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275541 5030 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-db\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275555 5030 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-web-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275569 5030 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275583 5030 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-tls-assets\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275596 5030 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-kube-rbac-proxy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275610 5030 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.275632 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275624 5030 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.276093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275638 5030 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.276093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275654 5030 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.276093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275668 5030 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-metrics-client-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.276093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.275682 5030 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.276412 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.276382 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:04.277644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.277611 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.277760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.277697 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "kube-api-access-54x9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:35:04.278339 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.278262 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.278819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.278745 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config" pod "46b438e0-fc10-4ba5-be8d-24f985295624" (UID: "46b438e0-fc10-4ba5-be8d-24f985295624"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:04.376254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.376167 5030 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-grpc-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.376254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.376198 5030 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/46b438e0-fc10-4ba5-be8d-24f985295624-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.376254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.376208 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54x9m\" (UniqueName: \"kubernetes.io/projected/46b438e0-fc10-4ba5-be8d-24f985295624-kube-api-access-54x9m\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.376254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.376219 5030 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.376254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.376230 5030 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/46b438e0-fc10-4ba5-be8d-24f985295624-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:04.836781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.836750 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" exitCode=0 Oct 29 22:35:04.836781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.836774 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" exitCode=0 Oct 29 22:35:04.836781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.836782 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" exitCode=0 Oct 29 22:35:04.836781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.836788 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" exitCode=0 Oct 29 22:35:04.836781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.836793 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" exitCode=0 Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.836798 5030 generic.go:356] "Generic (PLEG): container finished" podID="46b438e0-fc10-4ba5-be8d-24f985295624" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" exitCode=0 Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837026 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837051 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837158 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837204 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837225 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837242 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837278 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837320 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} Oct 29 22:35:04.837475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.837360 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"46b438e0-fc10-4ba5-be8d-24f985295624","Type":"ContainerDied","Data":"29cead7bc12b86a79afb29ebe3419efda88fd56eb1db4701ccfcd98a0f46a961"} Oct 29 22:35:04.838607 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.838297 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" event={"ID":"b6e248fe-a23c-46ed-b15d-6b318912801d","Type":"ContainerStarted","Data":"c3b7fbfd8281b5af7a0c19ee809b84d4845c15884b23bbc3e91be5f8bda4ba36"} Oct 29 22:35:04.852153 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.852138 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.862098 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.862078 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.870774 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.870752 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.879741 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.879723 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.888027 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.887994 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.896819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.896804 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.904815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.904783 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.905270 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.905251 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.905350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.905281 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} err="failed to get container status \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" Oct 29 22:35:04.905350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.905305 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.905642 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.905598 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.905684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.905652 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} err="failed to get container status \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" Oct 29 22:35:04.905684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.905677 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.905982 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.905952 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.906107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.905991 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} err="failed to get container status \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" Oct 29 22:35:04.906107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.906016 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.906734 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.906711 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.906799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.906740 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} err="failed to get container status \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" Oct 29 22:35:04.906799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.906760 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.907113 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.907091 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.907167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.907115 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} err="failed to get container status \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" Oct 29 22:35:04.907167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.907131 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.907435 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.907410 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.907585 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.907439 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} err="failed to get container status \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" Oct 29 22:35:04.907585 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.907455 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.907706 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:04.907681 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.907764 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.907709 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} err="failed to get container status \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" Oct 29 22:35:04.907764 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.907729 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.908044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908009 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} err="failed to get container status \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" Oct 29 22:35:04.908044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908033 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.908269 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908252 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} err="failed to get container status \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" Oct 29 22:35:04.908269 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908267 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.908483 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908468 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} err="failed to get container status \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" Oct 29 22:35:04.908545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908483 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.908715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908681 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} err="failed to get container status \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" Oct 29 22:35:04.908770 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908717 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.909005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.908987 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} err="failed to get container status \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" Oct 29 22:35:04.909061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909006 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.909237 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909221 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} err="failed to get container status \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" Oct 29 22:35:04.909282 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909239 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.909477 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909459 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} err="failed to get container status \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" Oct 29 22:35:04.909520 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909478 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.909733 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909715 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} err="failed to get container status \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" Oct 29 22:35:04.909779 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909732 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.909967 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909952 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} err="failed to get container status \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" Oct 29 22:35:04.910028 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.909967 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.910218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.910199 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} err="failed to get container status \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" Oct 29 22:35:04.910218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.910215 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.910545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.910520 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} err="failed to get container status \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" Oct 29 22:35:04.910607 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.910547 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.910933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.910914 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} err="failed to get container status \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" Oct 29 22:35:04.910991 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.910933 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.911198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911181 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} err="failed to get container status \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" Oct 29 22:35:04.911241 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911198 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.911447 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911427 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} err="failed to get container status \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" Oct 29 22:35:04.911447 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911446 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.911689 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911673 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} err="failed to get container status \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" Oct 29 22:35:04.911736 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911688 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.911947 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911927 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} err="failed to get container status \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" Oct 29 22:35:04.911990 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.911949 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.912192 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.912171 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} err="failed to get container status \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" Oct 29 22:35:04.912268 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.912193 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.912511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.912491 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} err="failed to get container status \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" Oct 29 22:35:04.912590 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.912519 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.912826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.912807 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} err="failed to get container status \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" Oct 29 22:35:04.912924 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.912826 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.913033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913011 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:35:04.913125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913112 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} err="failed to get container status \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" Oct 29 22:35:04.913169 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913128 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.913361 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913346 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} err="failed to get container status \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" Oct 29 22:35:04.913415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913361 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.913611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913589 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} err="failed to get container status \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" Oct 29 22:35:04.913611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913611 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.913893 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913871 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} err="failed to get container status \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" Oct 29 22:35:04.913965 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.913894 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.914130 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914110 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} err="failed to get container status \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" Oct 29 22:35:04.914175 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914130 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.914407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914384 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} err="failed to get container status \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" Oct 29 22:35:04.914407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914408 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.914650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914634 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} err="failed to get container status \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" Oct 29 22:35:04.914705 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914650 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.914894 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914875 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} err="failed to get container status \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" Oct 29 22:35:04.914974 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.914896 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.915183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915156 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} err="failed to get container status \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" Oct 29 22:35:04.915183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915174 5030 scope.go:122] "RemoveContainer" containerID="b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26" Oct 29 22:35:04.915435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915402 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26"} err="failed to get container status \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": rpc error: code = NotFound desc = could not find container \"b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26\": container with ID starting with b4f4a314cec26be64c727b5ab7b56f9930a65cc527c02bb79cadca3d5310cb26 not found: ID does not exist" Oct 29 22:35:04.915472 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915438 5030 scope.go:122] "RemoveContainer" containerID="0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e" Oct 29 22:35:04.915696 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915673 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e"} err="failed to get container status \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": rpc error: code = NotFound desc = could not find container \"0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e\": container with ID starting with 0378f48994782fef33b915fb5a36fab3d2b15dc9d2f3ccd3aac69e9ad762720e not found: ID does not exist" Oct 29 22:35:04.915696 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915694 5030 scope.go:122] "RemoveContainer" containerID="48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9" Oct 29 22:35:04.915932 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915916 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9"} err="failed to get container status \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": rpc error: code = NotFound desc = could not find container \"48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9\": container with ID starting with 48e86ff0c447e1f8abb4a95ea312a2a61df2d2bf0e2e57d513255227b43ebea9 not found: ID does not exist" Oct 29 22:35:04.915932 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.915932 5030 scope.go:122] "RemoveContainer" containerID="cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe" Oct 29 22:35:04.916179 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916154 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe"} err="failed to get container status \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": rpc error: code = NotFound desc = could not find container \"cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe\": container with ID starting with cd94311c93cfe26073b68c99b5e5001c9bf9adfb0ae771a8d8a1377365bd7ebe not found: ID does not exist" Oct 29 22:35:04.916179 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916175 5030 scope.go:122] "RemoveContainer" containerID="4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417" Oct 29 22:35:04.916420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916403 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417"} err="failed to get container status \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": rpc error: code = NotFound desc = could not find container \"4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417\": container with ID starting with 4da0ff76e862036d292bf724266182056d40d4a9e25b764ae2bfb86471c30417 not found: ID does not exist" Oct 29 22:35:04.916459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916421 5030 scope.go:122] "RemoveContainer" containerID="85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b" Oct 29 22:35:04.916625 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916605 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b"} err="failed to get container status \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": rpc error: code = NotFound desc = could not find container \"85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b\": container with ID starting with 85345f56375fda46ff9197bb57f7e29cf123459a6bed231cedf1da918286f52b not found: ID does not exist" Oct 29 22:35:04.916665 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916626 5030 scope.go:122] "RemoveContainer" containerID="8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b" Oct 29 22:35:04.916834 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.916815 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b"} err="failed to get container status \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": rpc error: code = NotFound desc = could not find container \"8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b\": container with ID starting with 8829cb6b5d1890d1c3dee18ac11db9d36fcefd1720da8b1ec81e4a850b0d6d6b not found: ID does not exist" Oct 29 22:35:04.926893 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.926871 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:35:04.942158 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942136 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:35:04.942546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942534 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="thanos-sidecar" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942548 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="thanos-sidecar" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942563 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="prometheus" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942568 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="prometheus" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942575 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942580 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942586 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-thanos" Oct 29 22:35:04.942596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942591 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-thanos" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942603 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="config-reloader" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942608 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="config-reloader" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942619 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-web" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942624 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-web" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942632 5030 cpu_manager.go:414] "containerMap: removing container" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="init-config-reloader" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942636 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="init-config-reloader" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942698 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="thanos-sidecar" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942707 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="prometheus" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942712 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942716 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-thanos" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942722 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="config-reloader" Oct 29 22:35:04.942785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.942728 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" containerName="kube-rbac-proxy-web" Oct 29 22:35:04.948812 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.948792 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.967933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.967906 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Oct 29 22:35:04.968047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.967907 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Oct 29 22:35:04.968047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.967939 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Oct 29 22:35:04.968047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.967965 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Oct 29 22:35:04.968047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968003 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Oct 29 22:35:04.968047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968019 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Oct 29 22:35:04.968047 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968009 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Oct 29 22:35:04.968319 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968066 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b1501m0mlj688\"" Oct 29 22:35:04.968319 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968128 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zn2tp\"" Oct 29 22:35:04.968319 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968130 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Oct 29 22:35:04.968319 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968172 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Oct 29 22:35:04.968445 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968352 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Oct 29 22:35:04.968611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968598 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Oct 29 22:35:04.968806 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.968791 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Oct 29 22:35:04.981407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981368 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.981529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981438 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.981584 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981553 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.981731 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981710 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.981816 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981798 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-web-config\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.981950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981928 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982040 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.981965 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-config\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982040 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982030 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982087 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e8774a7-6423-4810-933a-705d058e884d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982132 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982247 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982154 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e8774a7-6423-4810-933a-705d058e884d-config-out\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982247 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982185 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982291 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982399 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982355 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982453 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982417 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982450 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982532 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982571 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzkmd\" (UniqueName: \"kubernetes.io/projected/6e8774a7-6423-4810-933a-705d058e884d-kube-api-access-tzkmd\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982638 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.982756 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.982665 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:04.984143 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.984121 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Oct 29 22:35:04.984232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.984145 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Oct 29 22:35:04.986660 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:04.986638 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:35:05.047611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.047579 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46b438e0-fc10-4ba5-be8d-24f985295624" path="/var/lib/kubelet/pods/46b438e0-fc10-4ba5-be8d-24f985295624/volumes" Oct 29 22:35:05.083454 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083426 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.083606 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083469 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e8774a7-6423-4810-933a-705d058e884d-config-out\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.083606 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083507 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.083606 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083540 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.083902 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083881 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.083985 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083926 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.083985 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.083970 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084080 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084011 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084080 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084038 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzkmd\" (UniqueName: \"kubernetes.io/projected/6e8774a7-6423-4810-933a-705d058e884d-kube-api-access-tzkmd\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084080 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084064 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084083 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084121 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084168 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084389 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084247 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084402 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084431 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-web-config\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084457 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084483 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-config\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084510 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.084599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.084556 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e8774a7-6423-4810-933a-705d058e884d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.085259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.085228 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.085498 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.085470 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.086591 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.086567 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.087284 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.087223 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.087383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.087279 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.087975 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.087949 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.088389 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.088367 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.088805 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.088783 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090045 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090017 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090148 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090043 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6e8774a7-6423-4810-933a-705d058e884d-config-out\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090392 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090369 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090495 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090480 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090555 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090515 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6e8774a7-6423-4810-933a-705d058e884d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090609 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090577 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.090898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.090865 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.091178 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.091147 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-config\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.091276 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.091228 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6e8774a7-6423-4810-933a-705d058e884d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.091582 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.091567 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.092307 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.092284 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6e8774a7-6423-4810-933a-705d058e884d-web-config\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.109379 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.109357 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzkmd\" (UniqueName: \"kubernetes.io/projected/6e8774a7-6423-4810-933a-705d058e884d-kube-api-access-tzkmd\") pod \"prometheus-k8s-0\" (UID: \"6e8774a7-6423-4810-933a-705d058e884d\") " pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.270788 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.270564 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:05.608506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.608482 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Oct 29 22:35:05.611360 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:35:05.611335 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8774a7_6423_4810_933a_705d058e884d.slice/crio-1569e66c80674a5347f31699996ec408afcd357b053075439cf6d139749f067e WatchSource:0}: Error finding container 1569e66c80674a5347f31699996ec408afcd357b053075439cf6d139749f067e: Status 404 returned error can't find the container with id 1569e66c80674a5347f31699996ec408afcd357b053075439cf6d139749f067e Oct 29 22:35:05.792924 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:05.792894 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:35:05.846032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.846007 5030 generic.go:356] "Generic (PLEG): container finished" podID="6e8774a7-6423-4810-933a-705d058e884d" containerID="b5852ff50e2cc4d09657879dde9d683293979440619bc9d5efa29d226e4a09b3" exitCode=0 Oct 29 22:35:05.846401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.846098 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerDied","Data":"b5852ff50e2cc4d09657879dde9d683293979440619bc9d5efa29d226e4a09b3"} Oct 29 22:35:05.846401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.846159 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"1569e66c80674a5347f31699996ec408afcd357b053075439cf6d139749f067e"} Oct 29 22:35:05.848576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.848557 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" event={"ID":"b6e248fe-a23c-46ed-b15d-6b318912801d","Type":"ContainerStarted","Data":"b797958cca3b271439c748a005018905a605c6e32e7bec3c8e2e15a2725c0294"} Oct 29 22:35:05.848664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.848606 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" event={"ID":"b6e248fe-a23c-46ed-b15d-6b318912801d","Type":"ContainerStarted","Data":"675bbc7140ba5cd42a25a7ad768d0eacae4ff2cfba690e1989a6351111a787d3"} Oct 29 22:35:05.848664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.848627 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" event={"ID":"b6e248fe-a23c-46ed-b15d-6b318912801d","Type":"ContainerStarted","Data":"a3e35c8639904f4f36de0b4cc2229dd11161a721a81078ec008aecfff5db419e"} Oct 29 22:35:05.948904 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:05.948818 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-58bd5db97b-9zfb8" podStartSLOduration=1.370984614 podStartE2EDuration="2.948804467s" totalImagesPullingTime="1.577819853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:35:05.945993938 +0000 UTC m=+103.058639546" watchObservedRunningTime="2026-10-29 22:35:05.948804467 +0000 UTC m=+103.061450052" Oct 29 22:35:06.168927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.168868 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-56d68db4b4-wzkfq"] Oct 29 22:35:06.172005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.171988 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.193624 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.193598 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d68db4b4-wzkfq"] Oct 29 22:35:06.300669 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.300635 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:35:06.305098 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305068 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305239 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305184 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305274 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305444 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305327 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305444 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305386 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305444 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305420 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdk9p\" (UniqueName: \"kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305486 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.305602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.305581 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.401725 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.401686 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:35:06.407208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407181 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407346 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407260 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407346 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407303 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407346 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407326 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407383 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407401 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdk9p\" (UniqueName: \"kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407429 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407455 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.407915 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.407727 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.408310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.408097 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.408455 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.408430 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.408548 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.408533 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.408776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.408539 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.410384 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.410364 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.410581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.410564 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.428949 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.428874 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdk9p\" (UniqueName: \"kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p\") pod \"console-56d68db4b4-wzkfq\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.488694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.488669 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:06.798357 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.798321 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d68db4b4-wzkfq"] Oct 29 22:35:06.803878 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:35:06.803828 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f219cbe_73d6_4ab9_a968_b652e29e58fe.slice/crio-42091a40099bdb453f08f29ab377ed6d883082eafff54aa2b99df5f6c58635e6 WatchSource:0}: Error finding container 42091a40099bdb453f08f29ab377ed6d883082eafff54aa2b99df5f6c58635e6: Status 404 returned error can't find the container with id 42091a40099bdb453f08f29ab377ed6d883082eafff54aa2b99df5f6c58635e6 Oct 29 22:35:06.858579 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.858551 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"b3f2f93d838d56153b8604254e7bc2daf27389056052176ab51c40747a3766f2"} Oct 29 22:35:06.859005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.858630 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"af0de7958422aa794e79a12e167b478a704f7620bc38e2b78835ece0c80f4023"} Oct 29 22:35:06.859005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.858675 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"9140ce529c7cc74f20b6c5ca7829ee8ea00e12079a42af1f9ee48412e859291b"} Oct 29 22:35:06.859005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.858716 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"e2cecd28887df981682c2b12befac5f50b3e090b7a2e49b0ff4d53f9f17777dd"} Oct 29 22:35:06.859005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.858755 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"c9be375717cfaabcaaab6fe10039abe0001a502d5adae23702ce74c5fc9da1b9"} Oct 29 22:35:06.859005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.858804 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6e8774a7-6423-4810-933a-705d058e884d","Type":"ContainerStarted","Data":"ea5fe01f09d61f914f9c1ec67f6cf2bddce8605a950b3ab47cac813c32b58d3c"} Oct 29 22:35:06.860016 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.859988 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d68db4b4-wzkfq" event={"ID":"2f219cbe-73d6-4ab9-a968-b652e29e58fe","Type":"ContainerStarted","Data":"42091a40099bdb453f08f29ab377ed6d883082eafff54aa2b99df5f6c58635e6"} Oct 29 22:35:06.877393 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.877367 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:35:06.912587 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.912533 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.896707356 podStartE2EDuration="2.912516551s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.809195ms" isStatefulPod=false podCreationTimestamp="2026-10-29 22:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:35:06.910532109 +0000 UTC m=+104.023177700" watchObservedRunningTime="2026-10-29 22:35:06.912516551 +0000 UTC m=+104.025162138" Oct 29 22:35:06.959664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:06.959626 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:35:07.866682 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:07.866645 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d68db4b4-wzkfq" event={"ID":"2f219cbe-73d6-4ab9-a968-b652e29e58fe","Type":"ContainerStarted","Data":"8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75"} Oct 29 22:35:07.911042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:07.910990 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-56d68db4b4-wzkfq" podStartSLOduration=1.910976358 podStartE2EDuration="1.910976358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:35:07.910195143 +0000 UTC m=+105.022840734" watchObservedRunningTime="2026-10-29 22:35:07.910976358 +0000 UTC m=+105.023621937" Oct 29 22:35:10.271359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:10.271327 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:35:12.372603 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:12.372570 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:35:16.489892 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:16.489863 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:16.490247 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:16.489919 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:16.494881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:16.494860 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:16.916250 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:16.916183 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:35:16.988383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:16.988350 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749797b86b-jk6qf"] Oct 29 22:35:35.793525 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:35.793442 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:35:42.016176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.016122 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-749797b86b-jk6qf" podUID="29c8a0f7-9903-4935-9a71-d0ce49c0996f" containerName="console" containerID="cri-o://24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf" gracePeriod=15 Oct 29 22:35:42.283692 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.283661 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:35:42.428275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428250 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt\" (UniqueName: \"kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428434 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428288 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428434 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428313 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428448 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428502 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428530 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428562 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:35:42.428615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428613 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428780 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428669 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle\") pod \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\" (UID: \"29c8a0f7-9903-4935-9a71-d0ce49c0996f\") " Oct 29 22:35:42.428780 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428719 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:42.428920 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428834 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:42.429033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.428924 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:42.429155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.429068 5030 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c8a0f7-9903-4935-9a71-d0ce49c0996f-tmp\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.429155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.429087 5030 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-oauth-serving-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.429155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.429100 5030 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.429155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.429113 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:35:42.430785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.430765 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:42.431223 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.431204 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "kube-api-access-6v4lt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:35:42.431223 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.431204 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert" pod "29c8a0f7-9903-4935-9a71-d0ce49c0996f" (UID: "29c8a0f7-9903-4935-9a71-d0ce49c0996f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:35:42.530525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.530468 5030 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.530525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.530492 5030 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-serving-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.530525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.530505 5030 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29c8a0f7-9903-4935-9a71-d0ce49c0996f-console-oauth-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.530525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.530518 5030 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29c8a0f7-9903-4935-9a71-d0ce49c0996f-trusted-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:42.530705 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:42.530530 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v4lt\" (UniqueName: \"kubernetes.io/projected/29c8a0f7-9903-4935-9a71-d0ce49c0996f-kube-api-access-6v4lt\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:35:43.037025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.036994 5030 generic.go:356] "Generic (PLEG): container finished" podID="29c8a0f7-9903-4935-9a71-d0ce49c0996f" containerID="24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf" exitCode=0 Oct 29 22:35:43.037415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.037036 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749797b86b-jk6qf" Oct 29 22:35:43.037415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.037095 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749797b86b-jk6qf" event={"ID":"29c8a0f7-9903-4935-9a71-d0ce49c0996f","Type":"ContainerDied","Data":"24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf"} Oct 29 22:35:43.037415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.037233 5030 scope.go:122] "RemoveContainer" containerID="24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf" Oct 29 22:35:43.043798 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.043772 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749797b86b-jk6qf" event={"ID":"29c8a0f7-9903-4935-9a71-d0ce49c0996f","Type":"ContainerDied","Data":"09d51a6b2f5fa4c010db06c38eb9e316cf4ea985a1305683fe940817d2947070"} Oct 29 22:35:43.047855 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.047824 5030 scope.go:122] "RemoveContainer" containerID="24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf" Oct 29 22:35:43.048142 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:35:43.048123 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf\": container with ID starting with 24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf not found: ID does not exist" containerID="24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf" Oct 29 22:35:43.048219 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.048149 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf"} err="failed to get container status \"24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf\": rpc error: code = NotFound desc = could not find container \"24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf\": container with ID starting with 24a1d52f6a22290f19b53d8613a3140bce4068cfbfb71da07fc07d66f08979cf not found: ID does not exist" Oct 29 22:35:43.096088 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.096064 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749797b86b-jk6qf"] Oct 29 22:35:43.109887 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:43.109864 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-749797b86b-jk6qf"] Oct 29 22:35:45.043980 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:35:45.043946 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29c8a0f7-9903-4935-9a71-d0ce49c0996f" path="/var/lib/kubelet/pods/29c8a0f7-9903-4935-9a71-d0ce49c0996f/volumes" Oct 29 22:36:05.271044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:05.271002 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:36:05.287864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:05.287826 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:36:05.792445 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:36:05.792410 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:36:06.168663 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:06.168582 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Oct 29 22:36:12.575619 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:12.575581 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:13.782031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:13.781992 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:13.872111 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:13.872084 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:26.433726 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:26.433676 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:26.523757 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:26.523724 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:29.855303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:29.855269 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:29.956555 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:29.956528 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:30.037066 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:30.037040 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:35.793313 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:36:35.793277 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:36:36.965303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:36.965271 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7595bd47b-dpvnl"] Oct 29 22:36:36.970507 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:36.970476 5030 cpu_manager.go:414] "containerMap: removing container" podUID="29c8a0f7-9903-4935-9a71-d0ce49c0996f" containerName="console" Oct 29 22:36:36.970507 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:36.970508 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29c8a0f7-9903-4935-9a71-d0ce49c0996f" containerName="console" Oct 29 22:36:36.970650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:36.970642 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="29c8a0f7-9903-4935-9a71-d0ce49c0996f" containerName="console" Oct 29 22:36:36.974045 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:36.974026 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:36.997134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:36.997112 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7595bd47b-dpvnl"] Oct 29 22:36:37.094633 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.094597 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-service-ca\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.094817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.094656 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-oauth-config\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.094817 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.094772 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-serving-cert\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.094969 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.094869 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pnhr\" (UniqueName: \"kubernetes.io/projected/a90bfb0c-6f10-431f-8686-3a8fdbf80718-kube-api-access-6pnhr\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.095024 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.094971 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a90bfb0c-6f10-431f-8686-3a8fdbf80718-tmp\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.095085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.095027 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-config\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.095085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.095053 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-oauth-serving-cert\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.095085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.095076 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-trusted-ca-bundle\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.099123 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.099097 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:37.196192 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196166 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-service-ca\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196342 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196221 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-oauth-config\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196342 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196258 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-serving-cert\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196342 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196299 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pnhr\" (UniqueName: \"kubernetes.io/projected/a90bfb0c-6f10-431f-8686-3a8fdbf80718-kube-api-access-6pnhr\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196492 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196446 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a90bfb0c-6f10-431f-8686-3a8fdbf80718-tmp\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196499 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-config\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196527 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-oauth-serving-cert\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196659 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196550 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-trusted-ca-bundle\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.196892 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.196866 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a90bfb0c-6f10-431f-8686-3a8fdbf80718-tmp\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.197026 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.197006 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-service-ca\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.197178 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.197163 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-oauth-serving-cert\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.197475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.197451 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-config\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.197672 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.197652 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a90bfb0c-6f10-431f-8686-3a8fdbf80718-trusted-ca-bundle\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.199105 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.199082 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-oauth-config\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.199201 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.199086 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a90bfb0c-6f10-431f-8686-3a8fdbf80718-console-serving-cert\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.200102 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.200079 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:37.219023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.219005 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pnhr\" (UniqueName: \"kubernetes.io/projected/a90bfb0c-6f10-431f-8686-3a8fdbf80718-kube-api-access-6pnhr\") pod \"console-7595bd47b-dpvnl\" (UID: \"a90bfb0c-6f10-431f-8686-3a8fdbf80718\") " pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.286245 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.286217 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:37.591697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:37.591673 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7595bd47b-dpvnl"] Oct 29 22:36:37.594410 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:36:37.594378 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda90bfb0c_6f10_431f_8686_3a8fdbf80718.slice/crio-4185d7dae68dc56964fdc5b2888276ef124ccf7b3b6074894d3b7096884ef13b WatchSource:0}: Error finding container 4185d7dae68dc56964fdc5b2888276ef124ccf7b3b6074894d3b7096884ef13b: Status 404 returned error can't find the container with id 4185d7dae68dc56964fdc5b2888276ef124ccf7b3b6074894d3b7096884ef13b Oct 29 22:36:38.308701 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:38.308670 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7595bd47b-dpvnl" event={"ID":"a90bfb0c-6f10-431f-8686-3a8fdbf80718","Type":"ContainerStarted","Data":"22f6fcebfcf72635a829f31f90560596357266a4fbd3769168de8d119f5b919c"} Oct 29 22:36:38.309093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:38.308719 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7595bd47b-dpvnl" event={"ID":"a90bfb0c-6f10-431f-8686-3a8fdbf80718","Type":"ContainerStarted","Data":"4185d7dae68dc56964fdc5b2888276ef124ccf7b3b6074894d3b7096884ef13b"} Oct 29 22:36:38.352106 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:38.352062 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7595bd47b-dpvnl" podStartSLOduration=2.352049624 podStartE2EDuration="2.352049624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:36:38.350575014 +0000 UTC m=+195.463220618" watchObservedRunningTime="2026-10-29 22:36:38.352049624 +0000 UTC m=+195.464695274" Oct 29 22:36:38.777500 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:38.777464 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:39.497538 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:39.497493 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:47.286518 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:47.286477 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:47.287071 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:47.286539 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:47.291225 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:47.291204 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:47.339021 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:47.338987 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:36:47.356997 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:47.356971 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7595bd47b-dpvnl" Oct 29 22:36:47.432585 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:36:47.432558 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d68db4b4-wzkfq"] Oct 29 22:37:05.793607 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:37:05.793560 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:37:12.461328 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.461274 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-56d68db4b4-wzkfq" podUID="2f219cbe-73d6-4ab9-a968-b652e29e58fe" containerName="console" containerID="cri-o://8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75" gracePeriod=15 Oct 29 22:37:12.647418 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.647388 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:37:12.706499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.706474 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:37:12.803650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803617 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.803820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803661 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.803820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803693 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.803977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803822 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.803977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803889 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.803977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803924 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.803977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803972 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p\" (UniqueName: \"kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.804183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.803982 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:37:12.804183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804139 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle\") pod \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\" (UID: \"2f219cbe-73d6-4ab9-a968-b652e29e58fe\") " Oct 29 22:37:12.804288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804194 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:37:12.804288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804223 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:37:12.804288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804278 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:37:12.804642 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804618 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:37:12.804898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804877 5030 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.804962 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804907 5030 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f219cbe-73d6-4ab9-a968-b652e29e58fe-tmp\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.804962 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804920 5030 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.804962 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804933 5030 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-oauth-serving-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.804962 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.804949 5030 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f219cbe-73d6-4ab9-a968-b652e29e58fe-trusted-ca-bundle\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.806014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.805993 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:37:12.806163 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.806145 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "kube-api-access-qdk9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:37:12.806364 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.806342 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert" pod "2f219cbe-73d6-4ab9-a968-b652e29e58fe" (UID: "2f219cbe-73d6-4ab9-a968-b652e29e58fe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:37:12.906085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.906061 5030 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-serving-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.906085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.906086 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdk9p\" (UniqueName: \"kubernetes.io/projected/2f219cbe-73d6-4ab9-a968-b652e29e58fe-kube-api-access-qdk9p\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:12.906223 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:12.906095 5030 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f219cbe-73d6-4ab9-a968-b652e29e58fe-console-oauth-config\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:37:13.476653 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.476622 5030 generic.go:356] "Generic (PLEG): container finished" podID="2f219cbe-73d6-4ab9-a968-b652e29e58fe" containerID="8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75" exitCode=0 Oct 29 22:37:13.477104 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.476677 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d68db4b4-wzkfq" Oct 29 22:37:13.477104 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.476704 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d68db4b4-wzkfq" event={"ID":"2f219cbe-73d6-4ab9-a968-b652e29e58fe","Type":"ContainerDied","Data":"8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75"} Oct 29 22:37:13.477104 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.476772 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d68db4b4-wzkfq" event={"ID":"2f219cbe-73d6-4ab9-a968-b652e29e58fe","Type":"ContainerDied","Data":"42091a40099bdb453f08f29ab377ed6d883082eafff54aa2b99df5f6c58635e6"} Oct 29 22:37:13.477104 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.476792 5030 scope.go:122] "RemoveContainer" containerID="8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75" Oct 29 22:37:13.485522 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.485508 5030 scope.go:122] "RemoveContainer" containerID="8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75" Oct 29 22:37:13.485793 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:37:13.485773 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75\": container with ID starting with 8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75 not found: ID does not exist" containerID="8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75" Oct 29 22:37:13.485864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.485800 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75"} err="failed to get container status \"8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75\": rpc error: code = NotFound desc = could not find container \"8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75\": container with ID starting with 8ec621b96482d86f89d116e56163d6cc5beb6a01f14808401cfc2cd50c4aaf75 not found: ID does not exist" Oct 29 22:37:13.533446 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.533415 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d68db4b4-wzkfq"] Oct 29 22:37:13.547166 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:13.547140 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56d68db4b4-wzkfq"] Oct 29 22:37:15.043718 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:15.043688 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f219cbe-73d6-4ab9-a968-b652e29e58fe" path="/var/lib/kubelet/pods/2f219cbe-73d6-4ab9-a968-b652e29e58fe/volumes" Oct 29 22:37:22.681310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:37:22.681271 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:37:35.793767 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:37:35.793731 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:38:05.793492 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:38:05.793453 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:38:12.743663 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:12.743626 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:38:23.070280 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:23.070256 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:38:23.071535 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:23.071515 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:38:35.793247 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:38:35.793164 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:38:46.280678 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:46.280643 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:38:46.368816 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:46.368787 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:38:49.207044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:49.207011 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:38:49.298071 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:38:49.298036 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:39:05.398156 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:39:05.398116 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:39:05.485414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:39:05.485367 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:39:05.793211 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:39:05.793182 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:39:12.854482 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:39:12.854440 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:39:35.794083 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:39:35.794041 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:39:46.962333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:39:46.962298 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:40:05.794019 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:40:05.793984 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:40:12.975052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:40:12.975011 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:40:35.793605 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:40:35.793568 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:41:05.793734 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:41:05.793692 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:41:13.094869 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:41:13.094822 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:41:35.793944 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:41:35.793864 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:41:43.472376 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:41:43.472339 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:41:44.308696 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:41:44.308669 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:41:44.429096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:41:44.429057 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:41:44.538288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:41:44.538254 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:41:44.628688 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:41:44.628620 5030 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" limit=1024 Oct 29 22:42:05.793859 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:42:05.793811 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:42:35.793718 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:42:35.793680 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:43:05.793414 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:43:05.793373 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:43:23.103442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:43:23.103412 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:43:23.105945 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:43:23.103764 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:43:35.794036 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:43:35.793997 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:44:05.793513 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:44:05.793481 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:44:35.794404 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:44:35.794321 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:44:57.819215 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.819179 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m"] Oct 29 22:44:57.819656 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.819635 5030 cpu_manager.go:414] "containerMap: removing container" podUID="2f219cbe-73d6-4ab9-a968-b652e29e58fe" containerName="console" Oct 29 22:44:57.819656 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.819646 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f219cbe-73d6-4ab9-a968-b652e29e58fe" containerName="console" Oct 29 22:44:57.819725 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.819713 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f219cbe-73d6-4ab9-a968-b652e29e58fe" containerName="console" Oct 29 22:44:57.823223 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.823208 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.834508 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.834486 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-webhook-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.834613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.834524 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-apiservice-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.834656 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.834630 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-manifests\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.834701 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.834659 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qxmg\" (UniqueName: \"kubernetes.io/projected/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-kube-api-access-2qxmg\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.834701 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.834686 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-tmp\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.841216 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.834742 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-metrics-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.873105 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.873080 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m"] Oct 29 22:44:57.897814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.897785 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 29 22:44:57.898034 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.898017 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 29 22:44:57.898260 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.898245 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k79nx\"" Oct 29 22:44:57.898324 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.898283 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Oct 29 22:44:57.900116 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.900100 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 29 22:44:57.900187 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.900159 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 29 22:44:57.936658 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.936637 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-webhook-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.936776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.936667 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-apiservice-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.936776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.936711 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-manifests\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.936776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.936738 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qxmg\" (UniqueName: \"kubernetes.io/projected/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-kube-api-access-2qxmg\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.936776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.936764 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-tmp\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.937019 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.936795 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-metrics-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.937708 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.937678 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-manifests\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.939461 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.939437 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-metrics-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.939602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.939581 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-apiservice-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.939704 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.939687 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-tmp\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.939828 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.939812 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-webhook-cert\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:57.958390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:57.958370 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qxmg\" (UniqueName: \"kubernetes.io/projected/0cc1c950-de4e-4ce7-8ab5-20415c1e62ed-kube-api-access-2qxmg\") pod \"opendatahub-operator-controller-manager-675b698459-wwm4m\" (UID: \"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed\") " pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:58.144375 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:58.144309 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:44:58.514925 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:58.514902 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m"] Oct 29 22:44:58.517107 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:44:58.517083 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc1c950_de4e_4ce7_8ab5_20415c1e62ed.slice/crio-68c674e252e1fd9b7f9278eb3e39beebe2eb46130e472f4420193bf0d7d85dd3 WatchSource:0}: Error finding container 68c674e252e1fd9b7f9278eb3e39beebe2eb46130e472f4420193bf0d7d85dd3: Status 404 returned error can't find the container with id 68c674e252e1fd9b7f9278eb3e39beebe2eb46130e472f4420193bf0d7d85dd3 Oct 29 22:44:58.518829 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:58.518809 5030 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 22:44:58.700707 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:44:58.700674 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" event={"ID":"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed","Type":"ContainerStarted","Data":"68c674e252e1fd9b7f9278eb3e39beebe2eb46130e472f4420193bf0d7d85dd3"} Oct 29 22:45:01.719827 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:01.719793 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" event={"ID":"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed","Type":"ContainerStarted","Data":"cfc06e585314587effa4c139789a4adf99d394e876ddb07ac75e511f94f60855"} Oct 29 22:45:02.726151 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:02.726116 5030 generic.go:356] "Generic (PLEG): container finished" podID="0cc1c950-de4e-4ce7-8ab5-20415c1e62ed" containerID="cfc06e585314587effa4c139789a4adf99d394e876ddb07ac75e511f94f60855" exitCode=0 Oct 29 22:45:02.726559 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:02.726160 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" event={"ID":"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed","Type":"ContainerDied","Data":"cfc06e585314587effa4c139789a4adf99d394e876ddb07ac75e511f94f60855"} Oct 29 22:45:03.741401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:03.741364 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" event={"ID":"0cc1c950-de4e-4ce7-8ab5-20415c1e62ed","Type":"ContainerStarted","Data":"d59c058683217ad958902a6d86b7a8064f8d06ac86f248f80b381ecedee2347f"} Oct 29 22:45:03.741814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:03.741446 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:45:03.828919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:03.828871 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" podStartSLOduration=3.236565606 podStartE2EDuration="6.828857967s" totalImagesPullingTime="3.141347201s" totalInitContainerRuntime="450.94516ms" isStatefulPod=false podCreationTimestamp="2026-10-29 22:44:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:45:03.814045554 +0000 UTC m=+700.926691143" watchObservedRunningTime="2026-10-29 22:45:03.828857967 +0000 UTC m=+700.941503629" Oct 29 22:45:05.792936 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:45:05.792901 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:45:14.747964 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:14.747932 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-675b698459-wwm4m" Oct 29 22:45:19.194927 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.194893 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592"] Oct 29 22:45:19.203936 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.203918 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.221316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.221290 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592"] Oct 29 22:45:19.233654 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.233629 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 29 22:45:19.233793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.233654 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 29 22:45:19.233793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.233668 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 29 22:45:19.233793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.233664 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 29 22:45:19.233793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.233692 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wdpvc\"" Oct 29 22:45:19.324893 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.324860 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.325032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.324899 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.325032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.324981 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.325109 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.325038 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82tqf\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.325161 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.325140 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.325193 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.325178 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.325230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.325205 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.426645 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426622 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82tqf\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.426750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426658 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.426750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426690 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.426750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426724 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.426944 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426776 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.426944 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426894 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.427049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.426948 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.427468 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.427447 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.429247 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.429228 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.429338 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.429294 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.429338 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.429308 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.429599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.429442 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.448542 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.448493 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.448999 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.448983 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82tqf\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf\") pod \"istiod-openshift-gateway-7876bfcb74-8x592\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.516230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.516208 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:19.852563 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:19.852540 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592"] Oct 29 22:45:19.854104 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:45:19.854080 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6635b5f_54ae_43ff_ba7c_ceed450b21be.slice/crio-5dda3a5a76458a54667570a5221e04c70d87bf8619da11777567ff451461b642 WatchSource:0}: Error finding container 5dda3a5a76458a54667570a5221e04c70d87bf8619da11777567ff451461b642: Status 404 returned error can't find the container with id 5dda3a5a76458a54667570a5221e04c70d87bf8619da11777567ff451461b642 Oct 29 22:45:20.827586 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:20.827543 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" event={"ID":"c6635b5f-54ae-43ff-ba7c-ceed450b21be","Type":"ContainerStarted","Data":"5dda3a5a76458a54667570a5221e04c70d87bf8619da11777567ff451461b642"} Oct 29 22:45:22.176127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:22.176097 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:45:22.176374 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:22.176158 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:45:22.843186 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:22.843143 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" event={"ID":"c6635b5f-54ae-43ff-ba7c-ceed450b21be","Type":"ContainerStarted","Data":"6c6670ca94e1228dc2c93ab0dee06738350d81d6d85732185ff04bcd49ba827f"} Oct 29 22:45:22.843380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:22.843216 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:22.920227 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:22.920179 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" podStartSLOduration=1.600817231 podStartE2EDuration="3.92016673s" totalImagesPullingTime="2.319349499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:45:22.919425139 +0000 UTC m=+720.032070727" watchObservedRunningTime="2026-10-29 22:45:22.92016673 +0000 UTC m=+720.032812318" Oct 29 22:45:23.849786 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:45:23.849758 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:45:35.793107 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:45:35.793074 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:46:05.793652 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:46:05.793604 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:46:20.290243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.290211 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms"] Oct 29 22:46:20.294295 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.294270 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.316271 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.316245 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms"] Oct 29 22:46:20.328012 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.327990 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pwkhn\"" Oct 29 22:46:20.335292 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.335257 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Oct 29 22:46:20.339739 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.339713 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-tnntd"] Oct 29 22:46:20.344184 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.344164 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.364142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.364123 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-cbbvf\"" Oct 29 22:46:20.364142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.364134 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Oct 29 22:46:20.375058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.375035 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-tnntd"] Oct 29 22:46:20.378185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.378161 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfj8r\" (UniqueName: \"kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r\") pod \"llmisvc-controller-manager-869d5dcc5d-tqwms\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.378278 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.378205 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tqwms\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.479491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.479463 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfj8r\" (UniqueName: \"kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r\") pod \"llmisvc-controller-manager-869d5dcc5d-tqwms\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.479491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.479493 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tqwms\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.479676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.479620 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65a8235c-4125-4ca5-82fb-3f61078c8dbd-tls-certs\") pod \"model-serving-api-75bd89fdfc-tnntd\" (UID: \"65a8235c-4125-4ca5-82fb-3f61078c8dbd\") " pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.479676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.479659 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gcdp\" (UniqueName: \"kubernetes.io/projected/65a8235c-4125-4ca5-82fb-3f61078c8dbd-kube-api-access-5gcdp\") pod \"model-serving-api-75bd89fdfc-tnntd\" (UID: \"65a8235c-4125-4ca5-82fb-3f61078c8dbd\") " pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.482092 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.482069 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tqwms\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.517007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.516982 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfj8r\" (UniqueName: \"kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r\") pod \"llmisvc-controller-manager-869d5dcc5d-tqwms\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.581316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.581235 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65a8235c-4125-4ca5-82fb-3f61078c8dbd-tls-certs\") pod \"model-serving-api-75bd89fdfc-tnntd\" (UID: \"65a8235c-4125-4ca5-82fb-3f61078c8dbd\") " pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.581316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.581299 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gcdp\" (UniqueName: \"kubernetes.io/projected/65a8235c-4125-4ca5-82fb-3f61078c8dbd-kube-api-access-5gcdp\") pod \"model-serving-api-75bd89fdfc-tnntd\" (UID: \"65a8235c-4125-4ca5-82fb-3f61078c8dbd\") " pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.583984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.583959 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65a8235c-4125-4ca5-82fb-3f61078c8dbd-tls-certs\") pod \"model-serving-api-75bd89fdfc-tnntd\" (UID: \"65a8235c-4125-4ca5-82fb-3f61078c8dbd\") " pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.606676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.606647 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gcdp\" (UniqueName: \"kubernetes.io/projected/65a8235c-4125-4ca5-82fb-3f61078c8dbd-kube-api-access-5gcdp\") pod \"model-serving-api-75bd89fdfc-tnntd\" (UID: \"65a8235c-4125-4ca5-82fb-3f61078c8dbd\") " pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.612438 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.612408 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:20.658152 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.658115 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:20.926519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.926494 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms"] Oct 29 22:46:20.929876 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:46:20.929803 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3ad28ae6_cd6c_4fe2_a17a_14c216578d67.slice/crio-01978de39dced531126e3fed7f731a740563d7cf79f776a0915be99fbfd81015 WatchSource:0}: Error finding container 01978de39dced531126e3fed7f731a740563d7cf79f776a0915be99fbfd81015: Status 404 returned error can't find the container with id 01978de39dced531126e3fed7f731a740563d7cf79f776a0915be99fbfd81015 Oct 29 22:46:20.974777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:20.974750 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-tnntd"] Oct 29 22:46:20.976876 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:46:20.976850 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a8235c_4125_4ca5_82fb_3f61078c8dbd.slice/crio-bb52737d7b182e223ab2248b4257eb7c0c56c6a56fc15320570eb0b3a98864b0 WatchSource:0}: Error finding container bb52737d7b182e223ab2248b4257eb7c0c56c6a56fc15320570eb0b3a98864b0: Status 404 returned error can't find the container with id bb52737d7b182e223ab2248b4257eb7c0c56c6a56fc15320570eb0b3a98864b0 Oct 29 22:46:21.125851 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:21.125817 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" event={"ID":"3ad28ae6-cd6c-4fe2-a17a-14c216578d67","Type":"ContainerStarted","Data":"01978de39dced531126e3fed7f731a740563d7cf79f776a0915be99fbfd81015"} Oct 29 22:46:21.127018 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:21.126992 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" event={"ID":"65a8235c-4125-4ca5-82fb-3f61078c8dbd","Type":"ContainerStarted","Data":"bb52737d7b182e223ab2248b4257eb7c0c56c6a56fc15320570eb0b3a98864b0"} Oct 29 22:46:23.156221 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:23.156141 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" event={"ID":"65a8235c-4125-4ca5-82fb-3f61078c8dbd","Type":"ContainerStarted","Data":"13d44969586a908ed53ef27e30b38418b0577e4aaab382254beac203c2fca47b"} Oct 29 22:46:23.156675 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:23.156287 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:23.200599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:23.200549 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" podStartSLOduration=1.326729681 podStartE2EDuration="3.200534394s" totalImagesPullingTime="1.873804713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:46:23.199236569 +0000 UTC m=+780.311882158" watchObservedRunningTime="2026-10-29 22:46:23.200534394 +0000 UTC m=+780.313179985" Oct 29 22:46:24.007044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.007013 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592"] Oct 29 22:46:24.007253 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.007231 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" podUID="c6635b5f-54ae-43ff-ba7c-ceed450b21be" containerName="discovery" containerID="cri-o://6c6670ca94e1228dc2c93ab0dee06738350d81d6d85732185ff04bcd49ba827f" gracePeriod=30 Oct 29 22:46:24.162574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.162543 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" event={"ID":"3ad28ae6-cd6c-4fe2-a17a-14c216578d67","Type":"ContainerStarted","Data":"fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947"} Oct 29 22:46:24.163066 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.162654 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:46:24.164609 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.164586 5030 generic.go:356] "Generic (PLEG): container finished" podID="c6635b5f-54ae-43ff-ba7c-ceed450b21be" containerID="6c6670ca94e1228dc2c93ab0dee06738350d81d6d85732185ff04bcd49ba827f" exitCode=0 Oct 29 22:46:24.164730 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.164654 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" event={"ID":"c6635b5f-54ae-43ff-ba7c-ceed450b21be","Type":"ContainerDied","Data":"6c6670ca94e1228dc2c93ab0dee06738350d81d6d85732185ff04bcd49ba827f"} Oct 29 22:46:24.207196 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.207146 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" podStartSLOduration=1.456263094 podStartE2EDuration="4.207126716s" totalImagesPullingTime="2.750863622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:46:24.206402762 +0000 UTC m=+781.319048377" watchObservedRunningTime="2026-10-29 22:46:24.207126716 +0000 UTC m=+781.319772305" Oct 29 22:46:24.264109 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.264056 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:46:24.425078 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425050 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.425232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425136 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.425232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425158 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.425232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425185 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.425232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425205 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.425462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425244 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.425462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.425288 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf\") pod \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\" (UID: \"c6635b5f-54ae-43ff-ba7c-ceed450b21be\") " Oct 29 22:46:24.426141 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.426092 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:46:24.427502 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.427474 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:46:24.427629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.427602 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:46:24.427919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.427891 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:46:24.428028 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.427954 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:46:24.428070 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.428022 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "kube-api-access-82tqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:46:24.428070 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.428028 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts" pod "c6635b5f-54ae-43ff-ba7c-ceed450b21be" (UID: "c6635b5f-54ae-43ff-ba7c-ceed450b21be"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:46:24.527142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527086 5030 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6635b5f-54ae-43ff-ba7c-ceed450b21be-local-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:24.527142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527108 5030 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-dns-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:24.527142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527118 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:24.527142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527128 5030 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-kubeconfig\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:24.527142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527136 5030 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c6635b5f-54ae-43ff-ba7c-ceed450b21be-istio-csr-ca-configmap\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:24.527371 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527153 5030 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c6635b5f-54ae-43ff-ba7c-ceed450b21be-cacerts\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:24.527371 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:24.527163 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82tqf\" (UniqueName: \"kubernetes.io/projected/c6635b5f-54ae-43ff-ba7c-ceed450b21be-kube-api-access-82tqf\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:46:25.170624 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:25.170597 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" Oct 29 22:46:25.171049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:25.170632 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592" event={"ID":"c6635b5f-54ae-43ff-ba7c-ceed450b21be","Type":"ContainerDied","Data":"5dda3a5a76458a54667570a5221e04c70d87bf8619da11777567ff451461b642"} Oct 29 22:46:25.171049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:25.170686 5030 scope.go:122] "RemoveContainer" containerID="6c6670ca94e1228dc2c93ab0dee06738350d81d6d85732185ff04bcd49ba827f" Oct 29 22:46:25.229428 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:25.229402 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592"] Oct 29 22:46:25.243267 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:25.243246 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-8x592"] Oct 29 22:46:27.043402 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:27.043363 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6635b5f-54ae-43ff-ba7c-ceed450b21be" path="/var/lib/kubelet/pods/c6635b5f-54ae-43ff-ba7c-ceed450b21be/volumes" Oct 29 22:46:34.169340 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:34.169306 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-tnntd" Oct 29 22:46:35.794144 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:46:35.794107 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:46:55.174055 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:46:55.174023 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:47:05.794032 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:05.793992 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:47:23.153555 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.153527 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn"] Oct 29 22:47:23.153997 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.153983 5030 cpu_manager.go:414] "containerMap: removing container" podUID="c6635b5f-54ae-43ff-ba7c-ceed450b21be" containerName="discovery" Oct 29 22:47:23.154051 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.153998 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6635b5f-54ae-43ff-ba7c-ceed450b21be" containerName="discovery" Oct 29 22:47:23.154108 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.154095 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6635b5f-54ae-43ff-ba7c-ceed450b21be" containerName="discovery" Oct 29 22:47:23.157334 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.157307 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:23.200783 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.200751 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 29 22:47:23.200783 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.200751 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 29 22:47:23.200982 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.200937 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-f2prj\"" Oct 29 22:47:23.204041 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.204016 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn"] Oct 29 22:47:23.235216 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.235191 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lqwk\" (UniqueName: \"kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk\") pod \"limitador-operator-controller-manager-856dd584c5-wrnjn\" (UID: \"e52b8854-efc5-4a9c-85dd-aa4c16934b05\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:23.336505 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.336478 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lqwk\" (UniqueName: \"kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk\") pod \"limitador-operator-controller-manager-856dd584c5-wrnjn\" (UID: \"e52b8854-efc5-4a9c-85dd-aa4c16934b05\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:23.366138 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.366115 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lqwk\" (UniqueName: \"kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk\") pod \"limitador-operator-controller-manager-856dd584c5-wrnjn\" (UID: \"e52b8854-efc5-4a9c-85dd-aa4c16934b05\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:23.472269 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.472248 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:23.778430 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:23.778408 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn"] Oct 29 22:47:23.781071 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:23.781044 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52b8854_efc5_4a9c_85dd_aa4c16934b05.slice/crio-d857a1ee6e2706e14585f0dc377e6d9310878263fd511e87d4675b6e1856276a WatchSource:0}: Error finding container d857a1ee6e2706e14585f0dc377e6d9310878263fd511e87d4675b6e1856276a: Status 404 returned error can't find the container with id d857a1ee6e2706e14585f0dc377e6d9310878263fd511e87d4675b6e1856276a Oct 29 22:47:24.476786 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:24.476750 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" event={"ID":"e52b8854-efc5-4a9c-85dd-aa4c16934b05","Type":"ContainerStarted","Data":"d857a1ee6e2706e14585f0dc377e6d9310878263fd511e87d4675b6e1856276a"} Oct 29 22:47:25.828463 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:25.828428 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-6fh2m"] Oct 29 22:47:25.832148 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:25.832128 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:25.855808 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:25.855766 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-48gnc\"" Oct 29 22:47:25.858268 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:25.858245 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-6fh2m"] Oct 29 22:47:25.860285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:25.860259 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsjgc\" (UniqueName: \"kubernetes.io/projected/ce6d1137-ebfd-454a-8c9b-022babc856bf-kube-api-access-rsjgc\") pod \"authorino-operator-68d6dd76dc-6fh2m\" (UID: \"ce6d1137-ebfd-454a-8c9b-022babc856bf\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:25.961681 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:25.961660 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsjgc\" (UniqueName: \"kubernetes.io/projected/ce6d1137-ebfd-454a-8c9b-022babc856bf-kube-api-access-rsjgc\") pod \"authorino-operator-68d6dd76dc-6fh2m\" (UID: \"ce6d1137-ebfd-454a-8c9b-022babc856bf\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:26.012482 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.012458 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsjgc\" (UniqueName: \"kubernetes.io/projected/ce6d1137-ebfd-454a-8c9b-022babc856bf-kube-api-access-rsjgc\") pod \"authorino-operator-68d6dd76dc-6fh2m\" (UID: \"ce6d1137-ebfd-454a-8c9b-022babc856bf\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:26.145605 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.145552 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:26.462891 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.462863 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-6fh2m"] Oct 29 22:47:26.465654 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:26.465630 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6d1137_ebfd_454a_8c9b_022babc856bf.slice/crio-8837d09b0b141b7c91b1a4e5cd6178a5a7478c974fc66a27d5d7b8c3de0950e3 WatchSource:0}: Error finding container 8837d09b0b141b7c91b1a4e5cd6178a5a7478c974fc66a27d5d7b8c3de0950e3: Status 404 returned error can't find the container with id 8837d09b0b141b7c91b1a4e5cd6178a5a7478c974fc66a27d5d7b8c3de0950e3 Oct 29 22:47:26.499830 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.499781 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" event={"ID":"ce6d1137-ebfd-454a-8c9b-022babc856bf","Type":"ContainerStarted","Data":"8837d09b0b141b7c91b1a4e5cd6178a5a7478c974fc66a27d5d7b8c3de0950e3"} Oct 29 22:47:26.502118 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.502058 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" event={"ID":"e52b8854-efc5-4a9c-85dd-aa4c16934b05","Type":"ContainerStarted","Data":"975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728"} Oct 29 22:47:26.502252 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.502190 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:26.599150 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:26.599093 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" podStartSLOduration=1.502355245 podStartE2EDuration="3.599075411s" totalImagesPullingTime="2.096720166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:26.598436072 +0000 UTC m=+843.711081677" watchObservedRunningTime="2026-10-29 22:47:26.599075411 +0000 UTC m=+843.711721001" Oct 29 22:47:28.263606 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.263572 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9"] Oct 29 22:47:28.267499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.267481 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:28.314983 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.314955 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-l8z9v\"" Oct 29 22:47:28.316713 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.316694 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9"] Oct 29 22:47:28.321533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.321515 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Oct 29 22:47:28.384664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.384634 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khbfd\" (UniqueName: \"kubernetes.io/projected/50f1affa-2d7d-49a6-b925-eaaae50edbd3-kube-api-access-khbfd\") pod \"dns-operator-controller-manager-65bfd79598-vrxt9\" (UID: \"50f1affa-2d7d-49a6-b925-eaaae50edbd3\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:28.486087 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.486059 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khbfd\" (UniqueName: \"kubernetes.io/projected/50f1affa-2d7d-49a6-b925-eaaae50edbd3-kube-api-access-khbfd\") pod \"dns-operator-controller-manager-65bfd79598-vrxt9\" (UID: \"50f1affa-2d7d-49a6-b925-eaaae50edbd3\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:28.510506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.510481 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khbfd\" (UniqueName: \"kubernetes.io/projected/50f1affa-2d7d-49a6-b925-eaaae50edbd3-kube-api-access-khbfd\") pod \"dns-operator-controller-manager-65bfd79598-vrxt9\" (UID: \"50f1affa-2d7d-49a6-b925-eaaae50edbd3\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:28.515344 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.515275 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" event={"ID":"ce6d1137-ebfd-454a-8c9b-022babc856bf","Type":"ContainerStarted","Data":"7ecc5c4cb6adf57fc0845d36fb651f27ed806c02a8bdd14ccf56e22987d11d98"} Oct 29 22:47:28.515444 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.515357 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:28.577568 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.577515 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" podStartSLOduration=2.414845757 podStartE2EDuration="3.57749849s" totalImagesPullingTime="1.162652733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:28.577056548 +0000 UTC m=+845.689702136" watchObservedRunningTime="2026-10-29 22:47:28.57749849 +0000 UTC m=+845.690144079" Oct 29 22:47:28.580545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.580528 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:28.887887 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:28.887862 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9"] Oct 29 22:47:28.890539 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:28.890516 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f1affa_2d7d_49a6_b925_eaaae50edbd3.slice/crio-0dd8f811514cf604043add4c01d46886fa822d57e5cbe2645f454d28f6a5af28 WatchSource:0}: Error finding container 0dd8f811514cf604043add4c01d46886fa822d57e5cbe2645f454d28f6a5af28: Status 404 returned error can't find the container with id 0dd8f811514cf604043add4c01d46886fa822d57e5cbe2645f454d28f6a5af28 Oct 29 22:47:29.524965 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:29.524891 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" event={"ID":"50f1affa-2d7d-49a6-b925-eaaae50edbd3","Type":"ContainerStarted","Data":"0dd8f811514cf604043add4c01d46886fa822d57e5cbe2645f454d28f6a5af28"} Oct 29 22:47:31.537961 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:31.537931 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" event={"ID":"50f1affa-2d7d-49a6-b925-eaaae50edbd3","Type":"ContainerStarted","Data":"1b424191046a9d2f24776156a16f931a8185fcdd6858ff5283c067cab74dfe27"} Oct 29 22:47:31.538379 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:31.538042 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:31.592646 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:31.592601 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" podStartSLOduration=1.971447442 podStartE2EDuration="3.592589837s" totalImagesPullingTime="1.621142395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:31.590184985 +0000 UTC m=+848.702830572" watchObservedRunningTime="2026-10-29 22:47:31.592589837 +0000 UTC m=+848.705235425" Oct 29 22:47:33.146484 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.146456 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2"] Oct 29 22:47:33.150296 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.150278 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.158126 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.158102 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2"] Oct 29 22:47:33.170159 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.170137 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vwgmk\"" Oct 29 22:47:33.234656 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.234634 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.234789 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.234707 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngmwx\" (UniqueName: \"kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx\") pod \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.335974 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.335946 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.336114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.336041 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngmwx\" (UniqueName: \"kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx\") pod \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.336309 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.336290 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.370783 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.370752 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngmwx\" (UniqueName: \"kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx\") pod \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.464884 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.464773 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:33.795421 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:33.795401 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2"] Oct 29 22:47:33.797287 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:33.797263 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdae0073_cc20_4ca7_a8b7_6cc47c38a923.slice/crio-206cd4fb338ac5f84c9ec07eee6bee5905e45b918b51291a5b6d604aba754be7 WatchSource:0}: Error finding container 206cd4fb338ac5f84c9ec07eee6bee5905e45b918b51291a5b6d604aba754be7: Status 404 returned error can't find the container with id 206cd4fb338ac5f84c9ec07eee6bee5905e45b918b51291a5b6d604aba754be7 Oct 29 22:47:34.559433 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:34.559369 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" event={"ID":"bdae0073-cc20-4ca7-a8b7-6cc47c38a923","Type":"ContainerStarted","Data":"206cd4fb338ac5f84c9ec07eee6bee5905e45b918b51291a5b6d604aba754be7"} Oct 29 22:47:35.799517 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:35.799477 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:47:37.510697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:37.510629 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:37.580199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:37.580162 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" event={"ID":"bdae0073-cc20-4ca7-a8b7-6cc47c38a923","Type":"ContainerStarted","Data":"4ccfb203ef873af6be7d148d1ceca7cc7c155f663886daf7189665ca25303e05"} Oct 29 22:47:37.580350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:37.580296 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:37.631482 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:37.631436 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" podStartSLOduration=1.186431944 podStartE2EDuration="4.631422062s" totalImagesPullingTime="3.444990118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:37.628994138 +0000 UTC m=+854.741639740" watchObservedRunningTime="2026-10-29 22:47:37.631422062 +0000 UTC m=+854.744067650" Oct 29 22:47:39.527721 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:39.527693 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-6fh2m" Oct 29 22:47:40.648771 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.648743 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66"] Oct 29 22:47:40.652974 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.652955 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.670298 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.670274 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Oct 29 22:47:40.671185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.671160 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66"] Oct 29 22:47:40.683478 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.683456 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Oct 29 22:47:40.683578 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.683466 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sxswg\"" Oct 29 22:47:40.813025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.812999 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrzq8\" (UniqueName: \"kubernetes.io/projected/3f1c6a88-9498-46a4-9df4-3916e42520e6-kube-api-access-wrzq8\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.813177 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.813132 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3f1c6a88-9498-46a4-9df4-3916e42520e6-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.813177 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.813162 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f1c6a88-9498-46a4-9df4-3916e42520e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.913993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.913925 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrzq8\" (UniqueName: \"kubernetes.io/projected/3f1c6a88-9498-46a4-9df4-3916e42520e6-kube-api-access-wrzq8\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.913993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.913991 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3f1c6a88-9498-46a4-9df4-3916e42520e6-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.914183 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.914011 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f1c6a88-9498-46a4-9df4-3916e42520e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.914641 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.914617 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3f1c6a88-9498-46a4-9df4-3916e42520e6-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.916520 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.916499 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f1c6a88-9498-46a4-9df4-3916e42520e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.935643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.935621 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrzq8\" (UniqueName: \"kubernetes.io/projected/3f1c6a88-9498-46a4-9df4-3916e42520e6-kube-api-access-wrzq8\") pod \"kuadrant-console-plugin-66c4fb575d-6md66\" (UID: \"3f1c6a88-9498-46a4-9df4-3916e42520e6\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:40.977583 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:40.977563 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" Oct 29 22:47:41.274066 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:41.274045 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66"] Oct 29 22:47:41.276503 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:41.276474 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f1c6a88_9498_46a4_9df4_3916e42520e6.slice/crio-9b28819cbf8d40255e926af7525f495c28c89626b55f56ecface46355b8874de WatchSource:0}: Error finding container 9b28819cbf8d40255e926af7525f495c28c89626b55f56ecface46355b8874de: Status 404 returned error can't find the container with id 9b28819cbf8d40255e926af7525f495c28c89626b55f56ecface46355b8874de Oct 29 22:47:41.610797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:41.610708 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" event={"ID":"3f1c6a88-9498-46a4-9df4-3916e42520e6","Type":"ContainerStarted","Data":"9b28819cbf8d40255e926af7525f495c28c89626b55f56ecface46355b8874de"} Oct 29 22:47:42.547337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:42.547310 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-vrxt9" Oct 29 22:47:46.646642 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:46.646608 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" event={"ID":"3f1c6a88-9498-46a4-9df4-3916e42520e6","Type":"ContainerStarted","Data":"eba2b03783b2edfd7ab58fdef6d1dfaffdc93ee6b8a2da9758d6d5bcd2171b86"} Oct 29 22:47:46.691269 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:46.691218 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-6md66" podStartSLOduration=2.223658064 podStartE2EDuration="6.691204174s" totalImagesPullingTime="4.46754611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:46.689578593 +0000 UTC m=+863.802224180" watchObservedRunningTime="2026-10-29 22:47:46.691204174 +0000 UTC m=+863.803849767" Oct 29 22:47:48.594377 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:48.594349 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:50.364382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.364341 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn"] Oct 29 22:47:50.368645 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.368624 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.402475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.402453 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn"] Oct 29 22:47:50.519333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.519299 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8q7l\" (UniqueName: \"kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l\") pod \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.519504 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.519356 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.620612 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.620530 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8q7l\" (UniqueName: \"kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l\") pod \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.620612 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.620609 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.621096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.621076 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.643422 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.643398 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8q7l\" (UniqueName: \"kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l\") pod \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:50.681480 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:50.681456 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:51.019638 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:51.019580 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn"] Oct 29 22:47:51.022061 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:51.022029 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fc93ba_c7d5_44bb_ba65_45429450e05d.slice/crio-41cc826560179e2677e0aec7a511c88f51b96297ec7342919c2e07f8c8b6fa89 WatchSource:0}: Error finding container 41cc826560179e2677e0aec7a511c88f51b96297ec7342919c2e07f8c8b6fa89: Status 404 returned error can't find the container with id 41cc826560179e2677e0aec7a511c88f51b96297ec7342919c2e07f8c8b6fa89 Oct 29 22:47:51.675051 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:51.675010 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" event={"ID":"22fc93ba-c7d5-44bb-ba65-45429450e05d","Type":"ContainerStarted","Data":"fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd"} Oct 29 22:47:51.675418 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:51.675110 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" event={"ID":"22fc93ba-c7d5-44bb-ba65-45429450e05d","Type":"ContainerStarted","Data":"41cc826560179e2677e0aec7a511c88f51b96297ec7342919c2e07f8c8b6fa89"} Oct 29 22:47:51.675418 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:51.675136 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:52.506079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.506031 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" podStartSLOduration=2.506015633 podStartE2EDuration="2.506015633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:51.759803789 +0000 UTC m=+868.872449376" watchObservedRunningTime="2026-10-29 22:47:52.506015633 +0000 UTC m=+869.618661224" Oct 29 22:47:52.506237 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.506160 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2"] Oct 29 22:47:52.506348 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.506327 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" containerName="manager" containerID="cri-o://4ccfb203ef873af6be7d148d1ceca7cc7c155f663886daf7189665ca25303e05" gracePeriod=2 Oct 29 22:47:52.521118 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.521092 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2"] Oct 29 22:47:52.522808 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.522761 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.531381 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.531356 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th"] Oct 29 22:47:52.531820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.531808 5030 cpu_manager.go:414] "containerMap: removing container" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" containerName="manager" Oct 29 22:47:52.531895 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.531821 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" containerName="manager" Oct 29 22:47:52.532041 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.532029 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" containerName="manager" Oct 29 22:47:52.535277 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.535260 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.540546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.540517 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-722th\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.540659 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.540556 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cs6m\" (UniqueName: \"kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m\") pod \"kuadrant-operator-controller-manager-6b6fb56846-722th\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.548519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.548495 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn"] Oct 29 22:47:52.551000 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.550967 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.552990 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.552968 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn"] Oct 29 22:47:52.570874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.570829 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th"] Oct 29 22:47:52.579174 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.579150 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn"] Oct 29 22:47:52.579398 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.579366 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" containerName="manager" containerID="cri-o://975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728" gracePeriod=2 Oct 29 22:47:52.581366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.581332 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8"] Oct 29 22:47:52.582030 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.582010 5030 cpu_manager.go:414] "containerMap: removing container" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" containerName="manager" Oct 29 22:47:52.582030 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.582030 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" containerName="manager" Oct 29 22:47:52.582202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.582180 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" containerName="manager" Oct 29 22:47:52.585925 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.585904 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.596085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.596059 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn"] Oct 29 22:47:52.619278 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.619252 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp"] Oct 29 22:47:52.619933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.619916 5030 cpu_manager.go:414] "containerMap: removing container" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" containerName="manager" Oct 29 22:47:52.620025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.619934 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" containerName="manager" Oct 29 22:47:52.620025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.620024 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" containerName="manager" Oct 29 22:47:52.623189 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.623169 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:47:52.629535 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.629511 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8"] Oct 29 22:47:52.636933 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.636905 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.642083 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.642053 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6ph\" (UniqueName: \"kubernetes.io/projected/8c4e0ed4-a353-41ca-bff4-eaef1d54137b-kube-api-access-kv6ph\") pod \"limitador-operator-controller-manager-856dd584c5-cqkmp\" (UID: \"8c4e0ed4-a353-41ca-bff4-eaef1d54137b\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:47:52.642187 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.642112 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-722th\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.642187 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.642170 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cs6m\" (UniqueName: \"kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m\") pod \"kuadrant-operator-controller-manager-6b6fb56846-722th\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.642283 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.642227 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqvlg\" (UniqueName: \"kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg\") pod \"kuadrant-operator-controller-manager-6b76784d48-qx9g8\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.642407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.642389 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-qx9g8\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.642595 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.642578 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-722th\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.653475 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.653453 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-wrnjn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:52.662881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.662572 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp"] Oct 29 22:47:52.673322 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.673297 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cs6m\" (UniqueName: \"kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m\") pod \"kuadrant-operator-controller-manager-6b6fb56846-722th\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.684482 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.684452 5030 generic.go:356] "Generic (PLEG): container finished" podID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" containerID="4ccfb203ef873af6be7d148d1ceca7cc7c155f663886daf7189665ca25303e05" exitCode=0 Oct 29 22:47:52.730564 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.730518 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.743078 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.743048 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-qx9g8\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.743226 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.743128 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6ph\" (UniqueName: \"kubernetes.io/projected/8c4e0ed4-a353-41ca-bff4-eaef1d54137b-kube-api-access-kv6ph\") pod \"limitador-operator-controller-manager-856dd584c5-cqkmp\" (UID: \"8c4e0ed4-a353-41ca-bff4-eaef1d54137b\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:47:52.743226 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.743164 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqvlg\" (UniqueName: \"kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg\") pod \"kuadrant-operator-controller-manager-6b76784d48-qx9g8\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.743728 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.743704 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-qx9g8\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.746294 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.746267 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-wrnjn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:52.770058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.769427 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6ph\" (UniqueName: \"kubernetes.io/projected/8c4e0ed4-a353-41ca-bff4-eaef1d54137b-kube-api-access-kv6ph\") pod \"limitador-operator-controller-manager-856dd584c5-cqkmp\" (UID: \"8c4e0ed4-a353-41ca-bff4-eaef1d54137b\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:47:52.770058 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.770016 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqvlg\" (UniqueName: \"kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg\") pod \"kuadrant-operator-controller-manager-6b76784d48-qx9g8\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.820901 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.820881 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.825015 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.824996 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:52.836794 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.836773 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.844209 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.844186 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume\") pod \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " Oct 29 22:47:52.844294 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.844246 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk\" (UniqueName: \"kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk\") pod \"e52b8854-efc5-4a9c-85dd-aa4c16934b05\" (UID: \"e52b8854-efc5-4a9c-85dd-aa4c16934b05\") " Oct 29 22:47:52.844294 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.844265 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx\" (UniqueName: \"kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx\") pod \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\" (UID: \"bdae0073-cc20-4ca7-a8b7-6cc47c38a923\") " Oct 29 22:47:52.844670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.844646 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume" pod "bdae0073-cc20-4ca7-a8b7-6cc47c38a923" (UID: "bdae0073-cc20-4ca7-a8b7-6cc47c38a923"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:47:52.846496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.846480 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk" pod "e52b8854-efc5-4a9c-85dd-aa4c16934b05" (UID: "e52b8854-efc5-4a9c-85dd-aa4c16934b05"). InnerVolumeSpecName "kube-api-access-2lqwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:47:52.846536 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.846517 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx" pod "bdae0073-cc20-4ca7-a8b7-6cc47c38a923" (UID: "bdae0073-cc20-4ca7-a8b7-6cc47c38a923"). InnerVolumeSpecName "kube-api-access-ngmwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:47:52.852001 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.851974 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-wrnjn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:52.867566 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.867545 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-wrnjn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:52.882951 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:52.882927 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:52.944986 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.944964 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lqwk\" (UniqueName: \"kubernetes.io/projected/e52b8854-efc5-4a9c-85dd-aa4c16934b05-kube-api-access-2lqwk\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:47:52.944986 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.944983 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngmwx\" (UniqueName: \"kubernetes.io/projected/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-kube-api-access-ngmwx\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:47:52.945100 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.944992 5030 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdae0073-cc20-4ca7-a8b7-6cc47c38a923-extensions-socket-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:47:52.969202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.969172 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:52.975168 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.975148 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:52.995147 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:52.995126 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:47:53.056347 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.056316 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" path="/var/lib/kubelet/pods/bdae0073-cc20-4ca7-a8b7-6cc47c38a923/volumes" Oct 29 22:47:53.056676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.056663 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" path="/var/lib/kubelet/pods/e52b8854-efc5-4a9c-85dd-aa4c16934b05/volumes" Oct 29 22:47:53.063199 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:53.063165 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-dp8p2\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="bdae0073-cc20-4ca7-a8b7-6cc47c38a923" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:53.079864 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:53.078942 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-wrnjn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:53.386643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.386616 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp"] Oct 29 22:47:53.387342 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:53.387315 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4e0ed4_a353_41ca_bff4_eaef1d54137b.slice/crio-823aeaa6b207b0983ff38a75f38ae85eb25ad8dafcbc2a1657acf633343e5a91 WatchSource:0}: Error finding container 823aeaa6b207b0983ff38a75f38ae85eb25ad8dafcbc2a1657acf633343e5a91: Status 404 returned error can't find the container with id 823aeaa6b207b0983ff38a75f38ae85eb25ad8dafcbc2a1657acf633343e5a91 Oct 29 22:47:53.389212 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.389147 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th"] Oct 29 22:47:53.391568 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.391503 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8"] Oct 29 22:47:53.401005 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:53.400979 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56809d22_555e_45d0_9c2f_1fe460c6c328.slice/crio-8eb5d26729b926a2289241a62ae6c00cad9adcc505215c6852cf1250ae6254ca WatchSource:0}: Error finding container 8eb5d26729b926a2289241a62ae6c00cad9adcc505215c6852cf1250ae6254ca: Status 404 returned error can't find the container with id 8eb5d26729b926a2289241a62ae6c00cad9adcc505215c6852cf1250ae6254ca Oct 29 22:47:53.402828 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:47:53.402806 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e3fe59_409e_4078_a33e_afcbce7e9e16.slice/crio-038d2c674fbe6b794cfbf6e01ec4c5e07d967c09d45ffcba76eb4906bed1fdc4 WatchSource:0}: Error finding container 038d2c674fbe6b794cfbf6e01ec4c5e07d967c09d45ffcba76eb4906bed1fdc4: Status 404 returned error can't find the container with id 038d2c674fbe6b794cfbf6e01ec4c5e07d967c09d45ffcba76eb4906bed1fdc4 Oct 29 22:47:53.694499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.694425 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" event={"ID":"e7e3fe59-409e-4078-a33e-afcbce7e9e16","Type":"ContainerStarted","Data":"15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b"} Oct 29 22:47:53.694499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.694479 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" event={"ID":"e7e3fe59-409e-4078-a33e-afcbce7e9e16","Type":"ContainerStarted","Data":"038d2c674fbe6b794cfbf6e01ec4c5e07d967c09d45ffcba76eb4906bed1fdc4"} Oct 29 22:47:53.694986 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.694580 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:47:53.696051 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.696028 5030 generic.go:356] "Generic (PLEG): container finished" podID="e52b8854-efc5-4a9c-85dd-aa4c16934b05" containerID="975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728" exitCode=0 Oct 29 22:47:53.696188 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.696074 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-wrnjn" Oct 29 22:47:53.696188 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.696167 5030 scope.go:122] "RemoveContainer" containerID="975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728" Oct 29 22:47:53.697968 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.697908 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-dp8p2" Oct 29 22:47:53.700484 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.700368 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" event={"ID":"8c4e0ed4-a353-41ca-bff4-eaef1d54137b","Type":"ContainerStarted","Data":"f2167b4141597b3aa59a8b6e2569f99b5845c21dc95789e0c3b809a9d6d99471"} Oct 29 22:47:53.700484 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.700415 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" event={"ID":"8c4e0ed4-a353-41ca-bff4-eaef1d54137b","Type":"ContainerStarted","Data":"823aeaa6b207b0983ff38a75f38ae85eb25ad8dafcbc2a1657acf633343e5a91"} Oct 29 22:47:53.700484 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.700445 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:47:53.705911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.702764 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" event={"ID":"56809d22-555e-45d0-9c2f-1fe460c6c328","Type":"ContainerStarted","Data":"a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412"} Oct 29 22:47:53.705911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.702883 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:47:53.705911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.702925 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" event={"ID":"56809d22-555e-45d0-9c2f-1fe460c6c328","Type":"ContainerStarted","Data":"8eb5d26729b926a2289241a62ae6c00cad9adcc505215c6852cf1250ae6254ca"} Oct 29 22:47:53.705911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.703024 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" containerName="manager" containerID="cri-o://fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd" gracePeriod=2 Oct 29 22:47:53.707590 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.707562 5030 scope.go:122] "RemoveContainer" containerID="975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728" Oct 29 22:47:53.710224 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:53.710191 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728\": container with ID starting with 975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728 not found: ID does not exist" containerID="975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728" Oct 29 22:47:53.710324 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.710225 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728"} err="failed to get container status \"975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728\": rpc error: code = NotFound desc = could not find container \"975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728\": container with ID starting with 975a591bae8a3fd0054242bb1e1fbb597f0946a6d9d52749f1685b8bfcd40728 not found: ID does not exist" Oct 29 22:47:53.710324 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.710245 5030 scope.go:122] "RemoveContainer" containerID="4ccfb203ef873af6be7d148d1ceca7cc7c155f663886daf7189665ca25303e05" Oct 29 22:47:53.775875 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.770646 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" podStartSLOduration=1.770628468 podStartE2EDuration="1.770628468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:53.767269532 +0000 UTC m=+870.879915156" watchObservedRunningTime="2026-10-29 22:47:53.770628468 +0000 UTC m=+870.883274060" Oct 29 22:47:53.839414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.839372 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" podStartSLOduration=1.839358346 podStartE2EDuration="1.839358346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:53.836089727 +0000 UTC m=+870.948735317" watchObservedRunningTime="2026-10-29 22:47:53.839358346 +0000 UTC m=+870.952003933" Oct 29 22:47:53.887316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.887267 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" podStartSLOduration=1.887253942 podStartE2EDuration="1.887253942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:47:53.884905444 +0000 UTC m=+870.997551029" watchObservedRunningTime="2026-10-29 22:47:53.887253942 +0000 UTC m=+870.999899529" Oct 29 22:47:53.973229 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:53.973208 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:54.057506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.057479 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l\" (UniqueName: \"kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l\") pod \"22fc93ba-c7d5-44bb-ba65-45429450e05d\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " Oct 29 22:47:54.057655 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.057542 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume\") pod \"22fc93ba-c7d5-44bb-ba65-45429450e05d\" (UID: \"22fc93ba-c7d5-44bb-ba65-45429450e05d\") " Oct 29 22:47:54.057932 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.057906 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume" pod "22fc93ba-c7d5-44bb-ba65-45429450e05d" (UID: "22fc93ba-c7d5-44bb-ba65-45429450e05d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:47:54.059709 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.059688 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l" pod "22fc93ba-c7d5-44bb-ba65-45429450e05d" (UID: "22fc93ba-c7d5-44bb-ba65-45429450e05d"). InnerVolumeSpecName "kube-api-access-m8q7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:47:54.159214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.159184 5030 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22fc93ba-c7d5-44bb-ba65-45429450e05d-extensions-socket-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:47:54.159214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.159211 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8q7l\" (UniqueName: \"kubernetes.io/projected/22fc93ba-c7d5-44bb-ba65-45429450e05d-kube-api-access-m8q7l\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:47:54.710313 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.710279 5030 generic.go:356] "Generic (PLEG): container finished" podID="22fc93ba-c7d5-44bb-ba65-45429450e05d" containerID="fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd" exitCode=0 Oct 29 22:47:54.710737 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.710325 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:54.710737 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.710369 5030 scope.go:122] "RemoveContainer" containerID="fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd" Oct 29 22:47:54.720216 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.720200 5030 scope.go:122] "RemoveContainer" containerID="fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd" Oct 29 22:47:54.720479 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:54.720454 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd\": container with ID starting with fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd not found: ID does not exist" containerID="fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd" Oct 29 22:47:54.720522 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:54.720487 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd"} err="failed to get container status \"fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd\": rpc error: code = NotFound desc = could not find container \"fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd\": container with ID starting with fb0977405eb3a8266c7705810f8d5d9da66e3b3da80b34e1d931bf210e5230cd not found: ID does not exist" Oct 29 22:47:54.726031 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:54.726006 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:54.741553 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:47:54.741529 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b76784d48-t8hcn\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-t8hcn" Oct 29 22:47:55.044757 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:47:55.044728 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22fc93ba-c7d5-44bb-ba65-45429450e05d" path="/var/lib/kubelet/pods/22fc93ba-c7d5-44bb-ba65-45429450e05d/volumes" Oct 29 22:48:04.714548 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:04.714522 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:48:04.715028 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:04.714587 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:48:04.715028 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:04.714713 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-cqkmp" Oct 29 22:48:04.829626 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:04.829598 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th"] Oct 29 22:48:04.829801 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:04.829780 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" podUID="e7e3fe59-409e-4078-a33e-afcbce7e9e16" containerName="manager" containerID="cri-o://15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b" gracePeriod=10 Oct 29 22:48:05.100605 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.100583 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:48:05.164316 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.164287 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m\" (UniqueName: \"kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m\") pod \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " Oct 29 22:48:05.164472 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.164397 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume\") pod \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\" (UID: \"e7e3fe59-409e-4078-a33e-afcbce7e9e16\") " Oct 29 22:48:05.165397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.164829 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume" pod "e7e3fe59-409e-4078-a33e-afcbce7e9e16" (UID: "e7e3fe59-409e-4078-a33e-afcbce7e9e16"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:48:05.165397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.165018 5030 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7e3fe59-409e-4078-a33e-afcbce7e9e16-extensions-socket-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:48:05.167415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.167392 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m" pod "e7e3fe59-409e-4078-a33e-afcbce7e9e16" (UID: "e7e3fe59-409e-4078-a33e-afcbce7e9e16"). InnerVolumeSpecName "kube-api-access-5cs6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:48:05.227276 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.227246 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np"] Oct 29 22:48:05.227919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.227900 5030 cpu_manager.go:414] "containerMap: removing container" podUID="e7e3fe59-409e-4078-a33e-afcbce7e9e16" containerName="manager" Oct 29 22:48:05.227919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.227918 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7e3fe59-409e-4078-a33e-afcbce7e9e16" containerName="manager" Oct 29 22:48:05.228061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.228038 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7e3fe59-409e-4078-a33e-afcbce7e9e16" containerName="manager" Oct 29 22:48:05.231787 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.231772 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.257533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.257485 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np"] Oct 29 22:48:05.265705 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.265687 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cs6m\" (UniqueName: \"kubernetes.io/projected/e7e3fe59-409e-4078-a33e-afcbce7e9e16-kube-api-access-5cs6m\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:48:05.366700 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.366673 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn2m9\" (UniqueName: \"kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9\") pod \"kuadrant-operator-controller-manager-57bb9bfff-5g8np\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.366816 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.366714 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-5g8np\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.467751 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.467721 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2m9\" (UniqueName: \"kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9\") pod \"kuadrant-operator-controller-manager-57bb9bfff-5g8np\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.467919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.467767 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-5g8np\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.468178 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.468159 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-5g8np\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.501571 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.501545 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2m9\" (UniqueName: \"kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9\") pod \"kuadrant-operator-controller-manager-57bb9bfff-5g8np\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.548906 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.546965 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:05.773177 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.773147 5030 generic.go:356] "Generic (PLEG): container finished" podID="e7e3fe59-409e-4078-a33e-afcbce7e9e16" containerID="15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b" exitCode=0 Oct 29 22:48:05.773571 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.773203 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" Oct 29 22:48:05.773571 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.773232 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" event={"ID":"e7e3fe59-409e-4078-a33e-afcbce7e9e16","Type":"ContainerDied","Data":"15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b"} Oct 29 22:48:05.773571 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.773300 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th" event={"ID":"e7e3fe59-409e-4078-a33e-afcbce7e9e16","Type":"ContainerDied","Data":"038d2c674fbe6b794cfbf6e01ec4c5e07d967c09d45ffcba76eb4906bed1fdc4"} Oct 29 22:48:05.773571 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.773321 5030 scope.go:122] "RemoveContainer" containerID="15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b" Oct 29 22:48:05.782884 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.782861 5030 scope.go:122] "RemoveContainer" containerID="15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b" Oct 29 22:48:05.783181 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:48:05.783156 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b\": container with ID starting with 15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b not found: ID does not exist" containerID="15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b" Oct 29 22:48:05.783235 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.783189 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b"} err="failed to get container status \"15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b\": rpc error: code = NotFound desc = could not find container \"15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b\": container with ID starting with 15a83c7bfa077a657b0902b1d277d93687e1b26bab997b6b29840d54b85db36b not found: ID does not exist" Oct 29 22:48:05.794443 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:48:05.794420 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:48:05.834155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.834107 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th"] Oct 29 22:48:05.848662 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.848643 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-722th"] Oct 29 22:48:05.894185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:05.894166 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np"] Oct 29 22:48:05.896241 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:48:05.896217 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2712f7b_5e48_4f61_86e7_c4090f15d8de.slice/crio-4355f8f76d270e20b2fd33faec7f45ee29c9e235ca04c0933ecef3453b373ae8 WatchSource:0}: Error finding container 4355f8f76d270e20b2fd33faec7f45ee29c9e235ca04c0933ecef3453b373ae8: Status 404 returned error can't find the container with id 4355f8f76d270e20b2fd33faec7f45ee29c9e235ca04c0933ecef3453b373ae8 Oct 29 22:48:06.782571 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:06.782539 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" event={"ID":"c2712f7b-5e48-4f61-86e7-c4090f15d8de","Type":"ContainerStarted","Data":"299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3"} Oct 29 22:48:06.783079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:06.782592 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" event={"ID":"c2712f7b-5e48-4f61-86e7-c4090f15d8de","Type":"ContainerStarted","Data":"4355f8f76d270e20b2fd33faec7f45ee29c9e235ca04c0933ecef3453b373ae8"} Oct 29 22:48:06.783079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:06.782671 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:06.862434 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:06.862392 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" podStartSLOduration=1.862378993 podStartE2EDuration="1.862378993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:48:06.848223623 +0000 UTC m=+883.960869210" watchObservedRunningTime="2026-10-29 22:48:06.862378993 +0000 UTC m=+883.975024581" Oct 29 22:48:07.045523 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:07.045429 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7e3fe59-409e-4078-a33e-afcbce7e9e16" path="/var/lib/kubelet/pods/e7e3fe59-409e-4078-a33e-afcbce7e9e16/volumes" Oct 29 22:48:17.789800 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:17.789772 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 22:48:17.915354 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:17.915324 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8"] Oct 29 22:48:17.915645 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:17.915531 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" podUID="56809d22-555e-45d0-9c2f-1fe460c6c328" containerName="manager" containerID="cri-o://a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412" gracePeriod=10 Oct 29 22:48:18.221614 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.221587 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:48:18.393727 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.393659 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume\") pod \"56809d22-555e-45d0-9c2f-1fe460c6c328\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " Oct 29 22:48:18.393727 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.393716 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg\" (UniqueName: \"kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg\") pod \"56809d22-555e-45d0-9c2f-1fe460c6c328\" (UID: \"56809d22-555e-45d0-9c2f-1fe460c6c328\") " Oct 29 22:48:18.394108 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.394084 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume" pod "56809d22-555e-45d0-9c2f-1fe460c6c328" (UID: "56809d22-555e-45d0-9c2f-1fe460c6c328"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:48:18.395980 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.395957 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg" pod "56809d22-555e-45d0-9c2f-1fe460c6c328" (UID: "56809d22-555e-45d0-9c2f-1fe460c6c328"). InnerVolumeSpecName "kube-api-access-rqvlg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:48:18.495401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.495367 5030 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56809d22-555e-45d0-9c2f-1fe460c6c328-extensions-socket-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:48:18.495401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.495395 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqvlg\" (UniqueName: \"kubernetes.io/projected/56809d22-555e-45d0-9c2f-1fe460c6c328-kube-api-access-rqvlg\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:48:18.852534 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.852505 5030 generic.go:356] "Generic (PLEG): container finished" podID="56809d22-555e-45d0-9c2f-1fe460c6c328" containerID="a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412" exitCode=0 Oct 29 22:48:18.852941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.852552 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" Oct 29 22:48:18.852941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.852599 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" event={"ID":"56809d22-555e-45d0-9c2f-1fe460c6c328","Type":"ContainerDied","Data":"a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412"} Oct 29 22:48:18.852941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.852648 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8" event={"ID":"56809d22-555e-45d0-9c2f-1fe460c6c328","Type":"ContainerDied","Data":"8eb5d26729b926a2289241a62ae6c00cad9adcc505215c6852cf1250ae6254ca"} Oct 29 22:48:18.852941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.852681 5030 scope.go:122] "RemoveContainer" containerID="a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412" Oct 29 22:48:18.862575 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.862561 5030 scope.go:122] "RemoveContainer" containerID="a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412" Oct 29 22:48:18.862811 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:48:18.862797 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412\": container with ID starting with a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412 not found: ID does not exist" containerID="a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412" Oct 29 22:48:18.862895 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.862819 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412"} err="failed to get container status \"a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412\": rpc error: code = NotFound desc = could not find container \"a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412\": container with ID starting with a9a76bace1220e9282eb8870e0ca0c6d62d195d6415d68605d8ddba84abdb412 not found: ID does not exist" Oct 29 22:48:18.916050 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.916024 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8"] Oct 29 22:48:18.924699 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:18.924674 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-qx9g8"] Oct 29 22:48:19.044368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:19.044342 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56809d22-555e-45d0-9c2f-1fe460c6c328" path="/var/lib/kubelet/pods/56809d22-555e-45d0-9c2f-1fe460c6c328/volumes" Oct 29 22:48:22.005160 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.005122 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq"] Oct 29 22:48:22.005898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.005873 5030 cpu_manager.go:414] "containerMap: removing container" podUID="56809d22-555e-45d0-9c2f-1fe460c6c328" containerName="manager" Oct 29 22:48:22.005898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.005897 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56809d22-555e-45d0-9c2f-1fe460c6c328" containerName="manager" Oct 29 22:48:22.006153 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.006029 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="56809d22-555e-45d0-9c2f-1fe460c6c328" containerName="manager" Oct 29 22:48:22.016558 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.016538 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.031227 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.031202 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq"] Oct 29 22:48:22.042674 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.042653 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 29 22:48:22.043203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.043181 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z74sk\"" Oct 29 22:48:22.131033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131000 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa079846-a34e-4362-b007-8a85af7a852a-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131097 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131122 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131302 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131230 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131324 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131404 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131391 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131450 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131440 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdqrq\" (UniqueName: \"kubernetes.io/projected/fa079846-a34e-4362-b007-8a85af7a852a-kube-api-access-rdqrq\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131485 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131473 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa079846-a34e-4362-b007-8a85af7a852a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.131529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.131488 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa079846-a34e-4362-b007-8a85af7a852a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232172 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232349 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232240 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232349 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232274 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdqrq\" (UniqueName: \"kubernetes.io/projected/fa079846-a34e-4362-b007-8a85af7a852a-kube-api-access-rdqrq\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232349 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232314 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa079846-a34e-4362-b007-8a85af7a852a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232582 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232562 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa079846-a34e-4362-b007-8a85af7a852a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232633 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232602 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa079846-a34e-4362-b007-8a85af7a852a-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232625 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232637 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232653 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232678 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232734 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.232952 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.232898 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.233048 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.233025 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.233310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.233293 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa079846-a34e-4362-b007-8a85af7a852a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.234992 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.234966 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa079846-a34e-4362-b007-8a85af7a852a-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.234992 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.234994 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa079846-a34e-4362-b007-8a85af7a852a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.255243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.255189 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa079846-a34e-4362-b007-8a85af7a852a-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.255379 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.255357 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdqrq\" (UniqueName: \"kubernetes.io/projected/fa079846-a34e-4362-b007-8a85af7a852a-kube-api-access-rdqrq\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rt2xq\" (UID: \"fa079846-a34e-4362-b007-8a85af7a852a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.330067 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.330039 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:22.676008 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.675985 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq"] Oct 29 22:48:22.678014 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:48:22.677983 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa079846_a34e_4362_b007_8a85af7a852a.slice/crio-60d8fbe009cb501d9a80ee10c30f5396f61c4c1957901dca2dc570e190bb60d7 WatchSource:0}: Error finding container 60d8fbe009cb501d9a80ee10c30f5396f61c4c1957901dca2dc570e190bb60d7: Status 404 returned error can't find the container with id 60d8fbe009cb501d9a80ee10c30f5396f61c4c1957901dca2dc570e190bb60d7 Oct 29 22:48:22.876072 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:22.876045 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" event={"ID":"fa079846-a34e-4362-b007-8a85af7a852a","Type":"ContainerStarted","Data":"60d8fbe009cb501d9a80ee10c30f5396f61c4c1957901dca2dc570e190bb60d7"} Oct 29 22:48:23.140940 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:23.140815 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:48:23.155686 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:23.143544 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:48:25.448871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:25.448820 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:48:25.449121 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:25.448908 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:48:25.449121 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:25.448945 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:48:25.896724 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:25.896689 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" event={"ID":"fa079846-a34e-4362-b007-8a85af7a852a","Type":"ContainerStarted","Data":"ede89b4ec41e538d10c89bdc0fab2c2a03fb8cf959e4fbe7acec7a5e73e4690b"} Oct 29 22:48:26.331080 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:26.331049 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:26.335697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:26.335673 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:26.384420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:26.384377 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podStartSLOduration=2.616868996 podStartE2EDuration="5.384363319s" totalImagesPullingTime="2.767494323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:48:25.962287401 +0000 UTC m=+903.074932990" watchObservedRunningTime="2026-10-29 22:48:26.384363319 +0000 UTC m=+903.497008907" Oct 29 22:48:26.902702 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:26.902667 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:26.903460 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:26.903442 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:48:35.792995 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:48:35.792957 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:48:36.741355 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.741323 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:36.746478 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.746451 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:36.762790 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.762765 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:36.772549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.772524 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw4t4\" (UniqueName: \"kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4\") pod \"limitador-limitador-74dfcfb894-x5wx6\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:36.772685 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.772647 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file\") pod \"limitador-limitador-74dfcfb894-x5wx6\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:36.773993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.773974 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 29 22:48:36.873428 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.873384 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file\") pod \"limitador-limitador-74dfcfb894-x5wx6\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:36.873820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.873500 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw4t4\" (UniqueName: \"kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4\") pod \"limitador-limitador-74dfcfb894-x5wx6\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:36.874207 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.874187 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file\") pod \"limitador-limitador-74dfcfb894-x5wx6\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:36.879558 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.879535 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:36.900457 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:36.900429 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw4t4\" (UniqueName: \"kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4\") pod \"limitador-limitador-74dfcfb894-x5wx6\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:37.060327 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.060254 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:37.345105 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.345032 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:48:37.350133 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.350111 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.370998 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.370971 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:37.371120 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.371033 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:48:37.378198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.378170 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7773fbc0-77b8-4d48-94f5-7c3e18de77b3-config-file\") pod \"limitador-limitador-7c9b87d574-spfbm\" (UID: \"7773fbc0-77b8-4d48-94f5-7c3e18de77b3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.378320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.378220 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgjw9\" (UniqueName: \"kubernetes.io/projected/7773fbc0-77b8-4d48-94f5-7c3e18de77b3-kube-api-access-fgjw9\") pod \"limitador-limitador-7c9b87d574-spfbm\" (UID: \"7773fbc0-77b8-4d48-94f5-7c3e18de77b3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.436168 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.436142 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:48:37.479520 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.479496 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7773fbc0-77b8-4d48-94f5-7c3e18de77b3-config-file\") pod \"limitador-limitador-7c9b87d574-spfbm\" (UID: \"7773fbc0-77b8-4d48-94f5-7c3e18de77b3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.479668 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.479536 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgjw9\" (UniqueName: \"kubernetes.io/projected/7773fbc0-77b8-4d48-94f5-7c3e18de77b3-kube-api-access-fgjw9\") pod \"limitador-limitador-7c9b87d574-spfbm\" (UID: \"7773fbc0-77b8-4d48-94f5-7c3e18de77b3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.480266 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.480249 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7773fbc0-77b8-4d48-94f5-7c3e18de77b3-config-file\") pod \"limitador-limitador-7c9b87d574-spfbm\" (UID: \"7773fbc0-77b8-4d48-94f5-7c3e18de77b3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.501462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.501442 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgjw9\" (UniqueName: \"kubernetes.io/projected/7773fbc0-77b8-4d48-94f5-7c3e18de77b3-kube-api-access-fgjw9\") pod \"limitador-limitador-7c9b87d574-spfbm\" (UID: \"7773fbc0-77b8-4d48-94f5-7c3e18de77b3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.666190 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.666124 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:37.963914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.963816 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" event={"ID":"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263","Type":"ContainerStarted","Data":"f0cee17979c4bc9f221c7847880dfd03913de18c624749235d731b5b76e4697f"} Oct 29 22:48:37.969176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:37.969152 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:48:37.977296 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:48:37.977268 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7773fbc0_77b8_4d48_94f5_7c3e18de77b3.slice/crio-bfd97374cf45df98d4eaa10a90f45a32693041100bb91651d2e915b31a290d08 WatchSource:0}: Error finding container bfd97374cf45df98d4eaa10a90f45a32693041100bb91651d2e915b31a290d08: Status 404 returned error can't find the container with id bfd97374cf45df98d4eaa10a90f45a32693041100bb91651d2e915b31a290d08 Oct 29 22:48:38.970457 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:38.970417 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" event={"ID":"7773fbc0-77b8-4d48-94f5-7c3e18de77b3","Type":"ContainerStarted","Data":"9a7f53300f6a840ae0bbcada9c26f9d820ecc43773ee94250907c908c3a127a8"} Oct 29 22:48:38.970933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:38.970485 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:38.970933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:38.970510 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" event={"ID":"7773fbc0-77b8-4d48-94f5-7c3e18de77b3","Type":"ContainerStarted","Data":"bfd97374cf45df98d4eaa10a90f45a32693041100bb91651d2e915b31a290d08"} Oct 29 22:48:38.972029 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:38.972010 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" event={"ID":"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263","Type":"ContainerStarted","Data":"5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98"} Oct 29 22:48:38.972125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:38.972063 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:39.014277 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:39.014231 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" podStartSLOduration=1.523932534 podStartE2EDuration="2.014217827s" totalImagesPullingTime="490.285293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:48:39.013171271 +0000 UTC m=+916.125816881" watchObservedRunningTime="2026-10-29 22:48:39.014217827 +0000 UTC m=+916.126863409" Oct 29 22:48:39.058230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:39.058192 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" podStartSLOduration=2.170019756 podStartE2EDuration="3.058179047s" totalImagesPullingTime="888.159291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:48:39.058141951 +0000 UTC m=+916.170787540" watchObservedRunningTime="2026-10-29 22:48:39.058179047 +0000 UTC m=+916.170824801" Oct 29 22:48:49.978588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:49.978557 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:49.979063 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:49.978660 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-spfbm" Oct 29 22:48:50.096105 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.096068 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:50.096281 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.096246 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" podUID="f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" containerName="limitador" containerID="cri-o://5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98" gracePeriod=30 Oct 29 22:48:50.662742 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.662679 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:50.799011 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.798988 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file\" (UniqueName: \"kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file\") pod \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " Oct 29 22:48:50.799161 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.799111 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4\" (UniqueName: \"kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4\") pod \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\" (UID: \"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263\") " Oct 29 22:48:50.799326 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.799302 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file" pod "f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" (UID: "f47612d1-d58f-4f2a-9fc4-c18ed5f6c263"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:48:50.801466 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.801442 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4" pod "f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" (UID: "f47612d1-d58f-4f2a-9fc4-c18ed5f6c263"). InnerVolumeSpecName "kube-api-access-cw4t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:48:50.900508 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.900471 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw4t4\" (UniqueName: \"kubernetes.io/projected/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-kube-api-access-cw4t4\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:48:50.900508 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:50.900505 5030 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263-config-file\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:48:51.036726 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.036695 5030 generic.go:356] "Generic (PLEG): container finished" podID="f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" containerID="5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98" exitCode=0 Oct 29 22:48:51.037218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.036746 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" Oct 29 22:48:51.037218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.036781 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" event={"ID":"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263","Type":"ContainerDied","Data":"5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98"} Oct 29 22:48:51.041205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.037455 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x5wx6" event={"ID":"f47612d1-d58f-4f2a-9fc4-c18ed5f6c263","Type":"ContainerDied","Data":"f0cee17979c4bc9f221c7847880dfd03913de18c624749235d731b5b76e4697f"} Oct 29 22:48:51.041205 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.037479 5030 scope.go:122] "RemoveContainer" containerID="5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98" Oct 29 22:48:51.057425 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.057409 5030 scope.go:122] "RemoveContainer" containerID="5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98" Oct 29 22:48:51.057713 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:48:51.057693 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98\": container with ID starting with 5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98 not found: ID does not exist" containerID="5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98" Oct 29 22:48:51.057760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.057723 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98"} err="failed to get container status \"5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98\": rpc error: code = NotFound desc = could not find container \"5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98\": container with ID starting with 5c69a8ad6f687ebfe99564098235d68e501309b51c637c8ba89f978c36a22e98 not found: ID does not exist" Oct 29 22:48:51.095920 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.095889 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:51.109709 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:51.109685 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x5wx6"] Oct 29 22:48:53.044063 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:53.044027 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" path="/var/lib/kubelet/pods/f47612d1-d58f-4f2a-9fc4-c18ed5f6c263/volumes" Oct 29 22:48:55.022407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.022369 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d"] Oct 29 22:48:55.022900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.022878 5030 cpu_manager.go:414] "containerMap: removing container" podUID="f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" containerName="limitador" Oct 29 22:48:55.022900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.022890 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" containerName="limitador" Oct 29 22:48:55.023027 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.022978 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="f47612d1-d58f-4f2a-9fc4-c18ed5f6c263" containerName="limitador" Oct 29 22:48:55.027616 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.027595 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.046194 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.046168 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d"] Oct 29 22:48:55.055871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.055110 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 29 22:48:55.055871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.055117 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Oct 29 22:48:55.055871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.055199 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 29 22:48:55.058917 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.058896 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5x9fv\"" Oct 29 22:48:55.140617 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.140594 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzmk8\" (UniqueName: \"kubernetes.io/projected/bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97-kube-api-access-kzmk8\") pod \"postgres-7cdbf6dc44-gjp6d\" (UID: \"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.140794 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.140779 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97-data\") pod \"postgres-7cdbf6dc44-gjp6d\" (UID: \"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.241550 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.241520 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97-data\") pod \"postgres-7cdbf6dc44-gjp6d\" (UID: \"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.241690 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.241579 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzmk8\" (UniqueName: \"kubernetes.io/projected/bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97-kube-api-access-kzmk8\") pod \"postgres-7cdbf6dc44-gjp6d\" (UID: \"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.241926 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.241911 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97-data\") pod \"postgres-7cdbf6dc44-gjp6d\" (UID: \"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.263385 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.263362 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzmk8\" (UniqueName: \"kubernetes.io/projected/bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97-kube-api-access-kzmk8\") pod \"postgres-7cdbf6dc44-gjp6d\" (UID: \"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.340769 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.340713 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:48:55.644254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:55.644231 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d"] Oct 29 22:48:55.646432 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:48:55.646401 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5d07e1_6c1e_4c80_84a3_d5d8bfb60c97.slice/crio-84be9253bd80f49a13249d139b36874072c88bce8bfee918c136bd830f315289 WatchSource:0}: Error finding container 84be9253bd80f49a13249d139b36874072c88bce8bfee918c136bd830f315289: Status 404 returned error can't find the container with id 84be9253bd80f49a13249d139b36874072c88bce8bfee918c136bd830f315289 Oct 29 22:48:56.067320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:48:56.067291 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" event={"ID":"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97","Type":"ContainerStarted","Data":"84be9253bd80f49a13249d139b36874072c88bce8bfee918c136bd830f315289"} Oct 29 22:49:01.101813 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:01.101783 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" event={"ID":"bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97","Type":"ContainerStarted","Data":"af4769e8c61a0cc2becca7412008f760ed21927ee51ccdac9f3825301b4312c7"} Oct 29 22:49:01.102208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:01.101914 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:49:01.144993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:01.144951 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" podStartSLOduration=2.402156846 podStartE2EDuration="7.14493745s" totalImagesPullingTime="4.742780604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:49:01.143754968 +0000 UTC m=+938.256400551" watchObservedRunningTime="2026-10-29 22:49:01.14493745 +0000 UTC m=+938.257583228" Oct 29 22:49:05.793496 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:49:05.793459 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:49:07.138298 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:07.138272 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-gjp6d" Oct 29 22:49:16.635036 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.635006 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-rktf7"] Oct 29 22:49:16.639618 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.639600 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.656628 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.656605 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-rktf7"] Oct 29 22:49:16.668752 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.668731 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 29 22:49:16.671520 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.671504 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 29 22:49:16.671520 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.671515 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 29 22:49:16.671655 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.671624 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jmzz4\"" Oct 29 22:49:16.758243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.758218 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzqf\" (UniqueName: \"kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.758388 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.758370 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.758431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.758414 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.859200 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.859168 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzqf\" (UniqueName: \"kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.859353 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.859263 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.859353 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.859308 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.861746 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.861724 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.861861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.861760 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.879885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.879862 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzqf\" (UniqueName: \"kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf\") pod \"maas-controller-577f596567-rktf7\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.953342 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.953282 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:16.974619 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.974595 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-848f956548-92rkf"] Oct 29 22:49:16.980046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.980012 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:16.998090 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:16.998056 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-848f956548-92rkf"] Oct 29 22:49:17.060906 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.060882 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.061046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.061009 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.061114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.061068 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj6qq\" (UniqueName: \"kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.161964 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.161934 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.162124 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.161989 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj6qq\" (UniqueName: \"kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.162124 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.162055 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.164370 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.164345 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.164465 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.164391 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.183233 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.183207 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj6qq\" (UniqueName: \"kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq\") pod \"maas-controller-848f956548-92rkf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.260900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.260874 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-rktf7"] Oct 29 22:49:17.262687 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:49:17.262659 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9097c31e_b936_42cb_ae91_b319f3497609.slice/crio-e50f10f5f7884498cbf2b57b7f6c093aa80c926119e1bab2c000908247f424ed WatchSource:0}: Error finding container e50f10f5f7884498cbf2b57b7f6c093aa80c926119e1bab2c000908247f424ed: Status 404 returned error can't find the container with id e50f10f5f7884498cbf2b57b7f6c093aa80c926119e1bab2c000908247f424ed Oct 29 22:49:17.298570 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.298547 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:17.607849 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:17.607816 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-848f956548-92rkf"] Oct 29 22:49:17.611381 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:49:17.611355 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7883fd_ad17_4803_9278_3f05564850bf.slice/crio-98a868158d0a22a7a8de3cb8d24ec1bae6c424645de5fb6cfe23bca32a68a3da WatchSource:0}: Error finding container 98a868158d0a22a7a8de3cb8d24ec1bae6c424645de5fb6cfe23bca32a68a3da: Status 404 returned error can't find the container with id 98a868158d0a22a7a8de3cb8d24ec1bae6c424645de5fb6cfe23bca32a68a3da Oct 29 22:49:18.201669 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:18.201630 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-848f956548-92rkf" event={"ID":"fc7883fd-ad17-4803-9278-3f05564850bf","Type":"ContainerStarted","Data":"98a868158d0a22a7a8de3cb8d24ec1bae6c424645de5fb6cfe23bca32a68a3da"} Oct 29 22:49:18.204706 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:18.204675 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-rktf7" event={"ID":"9097c31e-b936-42cb-ae91-b319f3497609","Type":"ContainerStarted","Data":"e50f10f5f7884498cbf2b57b7f6c093aa80c926119e1bab2c000908247f424ed"} Oct 29 22:49:21.223815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:21.223784 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-848f956548-92rkf" event={"ID":"fc7883fd-ad17-4803-9278-3f05564850bf","Type":"ContainerStarted","Data":"e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f"} Oct 29 22:49:21.224288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:21.223972 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:21.225522 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:21.225499 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-rktf7" event={"ID":"9097c31e-b936-42cb-ae91-b319f3497609","Type":"ContainerStarted","Data":"8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1"} Oct 29 22:49:21.225630 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:21.225617 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:21.280398 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:21.280352 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-848f956548-92rkf" podStartSLOduration=2.648662172 podStartE2EDuration="5.28033999s" totalImagesPullingTime="2.631677818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:49:21.277695492 +0000 UTC m=+958.390341080" watchObservedRunningTime="2026-10-29 22:49:21.28033999 +0000 UTC m=+958.392985578" Oct 29 22:49:21.333790 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:21.333737 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-rktf7" podStartSLOduration=2.405553553 podStartE2EDuration="5.333724014s" totalImagesPullingTime="2.928170461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:49:21.331105437 +0000 UTC m=+958.443751024" watchObservedRunningTime="2026-10-29 22:49:21.333724014 +0000 UTC m=+958.446369633" Oct 29 22:49:23.933581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.933545 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp"] Oct 29 22:49:23.939459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.939439 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:23.958517 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.958493 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 29 22:49:23.958935 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.958649 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 29 22:49:23.958935 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.958730 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-5ldqc\"" Oct 29 22:49:23.959119 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.958892 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 29 22:49:23.960132 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:23.959559 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp"] Oct 29 22:49:24.045107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.045076 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/9a233e1b-cf6f-42a7-9141-5aded41232ce-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.045259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.045123 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a233e1b-cf6f-42a7-9141-5aded41232ce-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.045259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.045142 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a233e1b-cf6f-42a7-9141-5aded41232ce-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.045259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.045244 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdb68\" (UniqueName: \"kubernetes.io/projected/9a233e1b-cf6f-42a7-9141-5aded41232ce-kube-api-access-wdb68\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.146564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.146532 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a233e1b-cf6f-42a7-9141-5aded41232ce-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.146867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.146571 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a233e1b-cf6f-42a7-9141-5aded41232ce-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.146867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.146626 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdb68\" (UniqueName: \"kubernetes.io/projected/9a233e1b-cf6f-42a7-9141-5aded41232ce-kube-api-access-wdb68\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.146867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.146758 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/9a233e1b-cf6f-42a7-9141-5aded41232ce-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.147360 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.147341 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a233e1b-cf6f-42a7-9141-5aded41232ce-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.147492 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.147474 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/9a233e1b-cf6f-42a7-9141-5aded41232ce-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.149091 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.149072 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a233e1b-cf6f-42a7-9141-5aded41232ce-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.169602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.169583 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdb68\" (UniqueName: \"kubernetes.io/projected/9a233e1b-cf6f-42a7-9141-5aded41232ce-kube-api-access-wdb68\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-w4nsp\" (UID: \"9a233e1b-cf6f-42a7-9141-5aded41232ce\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.256685 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.256661 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:24.569621 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:24.569597 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp"] Oct 29 22:49:24.570754 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:49:24.570732 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a233e1b_cf6f_42a7_9141_5aded41232ce.slice/crio-85a0402122c0559b1feaa16f0c41b436fa7e582bd40f7107f27b08ffc8e043a3 WatchSource:0}: Error finding container 85a0402122c0559b1feaa16f0c41b436fa7e582bd40f7107f27b08ffc8e043a3: Status 404 returned error can't find the container with id 85a0402122c0559b1feaa16f0c41b436fa7e582bd40f7107f27b08ffc8e043a3 Oct 29 22:49:25.247154 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:25.247121 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" event={"ID":"9a233e1b-cf6f-42a7-9141-5aded41232ce","Type":"ContainerStarted","Data":"85a0402122c0559b1feaa16f0c41b436fa7e582bd40f7107f27b08ffc8e043a3"} Oct 29 22:49:32.233871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:32.233808 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 22:49:32.234311 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:32.234056 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:32.316750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:32.316718 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-rktf7"] Oct 29 22:49:32.317219 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:32.317159 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-rktf7" podUID="9097c31e-b936-42cb-ae91-b319f3497609" containerName="manager" containerID="cri-o://8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1" gracePeriod=30 Oct 29 22:49:33.177476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.177451 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:33.256552 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.256480 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs\") pod \"9097c31e-b936-42cb-ae91-b319f3497609\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " Oct 29 22:49:33.256552 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.256523 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert\") pod \"9097c31e-b936-42cb-ae91-b319f3497609\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " Oct 29 22:49:33.257079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.256676 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf\" (UniqueName: \"kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf\") pod \"9097c31e-b936-42cb-ae91-b319f3497609\" (UID: \"9097c31e-b936-42cb-ae91-b319f3497609\") " Oct 29 22:49:33.258859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.258812 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert" pod "9097c31e-b936-42cb-ae91-b319f3497609" (UID: "9097c31e-b936-42cb-ae91-b319f3497609"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:49:33.258970 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.258902 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs" pod "9097c31e-b936-42cb-ae91-b319f3497609" (UID: "9097c31e-b936-42cb-ae91-b319f3497609"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:49:33.258970 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.258907 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf" pod "9097c31e-b936-42cb-ae91-b319f3497609" (UID: "9097c31e-b936-42cb-ae91-b319f3497609"). InnerVolumeSpecName "kube-api-access-dmzqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:49:33.304768 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.304741 5030 generic.go:356] "Generic (PLEG): container finished" podID="9097c31e-b936-42cb-ae91-b319f3497609" containerID="8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1" exitCode=0 Oct 29 22:49:33.304886 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.304783 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-rktf7" Oct 29 22:49:33.304886 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.304785 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-rktf7" event={"ID":"9097c31e-b936-42cb-ae91-b319f3497609","Type":"ContainerDied","Data":"8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1"} Oct 29 22:49:33.304954 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.304889 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-rktf7" event={"ID":"9097c31e-b936-42cb-ae91-b319f3497609","Type":"ContainerDied","Data":"e50f10f5f7884498cbf2b57b7f6c093aa80c926119e1bab2c000908247f424ed"} Oct 29 22:49:33.304954 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.304915 5030 scope.go:122] "RemoveContainer" containerID="8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1" Oct 29 22:49:33.355668 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.355505 5030 scope.go:122] "RemoveContainer" containerID="8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1" Oct 29 22:49:33.356016 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:49:33.355990 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1\": container with ID starting with 8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1 not found: ID does not exist" containerID="8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1" Oct 29 22:49:33.356112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.356019 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1"} err="failed to get container status \"8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1\": rpc error: code = NotFound desc = could not find container \"8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1\": container with ID starting with 8bcaa6f4ec7d180e42f790ab65f35a31fe8a827f088269693a80b74b18aa07f1 not found: ID does not exist" Oct 29 22:49:33.357576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.357555 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmzqf\" (UniqueName: \"kubernetes.io/projected/9097c31e-b936-42cb-ae91-b319f3497609-kube-api-access-dmzqf\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:49:33.357576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.357576 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:49:33.357739 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.357584 5030 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9097c31e-b936-42cb-ae91-b319f3497609-webhook-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:49:33.362547 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.362502 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-rktf7"] Oct 29 22:49:33.364717 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:33.364698 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-rktf7"] Oct 29 22:49:35.045397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:35.045311 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9097c31e-b936-42cb-ae91-b319f3497609" path="/var/lib/kubelet/pods/9097c31e-b936-42cb-ae91-b319f3497609/volumes" Oct 29 22:49:35.319435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:35.319362 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" event={"ID":"9a233e1b-cf6f-42a7-9141-5aded41232ce","Type":"ContainerStarted","Data":"d3ab6fa907cda37ecb865facd071581a9f12cb4fe329a963ccbb10b9b25a8ff0"} Oct 29 22:49:35.319564 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:35.319478 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:35.364547 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:35.364503 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" podStartSLOduration=2.191946206 podStartE2EDuration="12.364491376s" totalImagesPullingTime="10.17254517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:49:35.362237773 +0000 UTC m=+972.474883378" watchObservedRunningTime="2026-10-29 22:49:35.364491376 +0000 UTC m=+972.477136964" Oct 29 22:49:35.794117 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:49:35.794076 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:49:46.331405 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:46.331372 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-w4nsp" Oct 29 22:49:54.313438 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.313403 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4"] Oct 29 22:49:54.314093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.314067 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9097c31e-b936-42cb-ae91-b319f3497609" containerName="manager" Oct 29 22:49:54.314093 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.314089 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9097c31e-b936-42cb-ae91-b319f3497609" containerName="manager" Oct 29 22:49:54.314273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.314184 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9097c31e-b936-42cb-ae91-b319f3497609" containerName="manager" Oct 29 22:49:54.320785 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.320768 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.335912 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.335892 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4"] Oct 29 22:49:54.347657 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.347636 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 29 22:49:54.347657 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.347636 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 29 22:49:54.351263 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.351236 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv74p\" (UniqueName: \"kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.351368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.351284 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.351456 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.351434 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.351528 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.351514 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.353038 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.353019 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7l5lq\"" Oct 29 22:49:54.452706 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.452676 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv74p\" (UniqueName: \"kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.452935 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.452728 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.452935 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.452798 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.452935 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.452892 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.453521 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.453500 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.455449 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.455429 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.455535 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.455479 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.473406 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.473385 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv74p\" (UniqueName: \"kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p\") pod \"maas-api-5c48877c75-wnnx4\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.637102 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.637038 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:54.941629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:54.941550 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4"] Oct 29 22:49:54.945681 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:49:54.945653 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2802b231_5ab0_43cd_85f1_c6c4c68c331a.slice/crio-d079ff1bdae3bc297dd28d49fd0dcfa89ab732b14e21dc1057ab23e1ab2b6caa WatchSource:0}: Error finding container d079ff1bdae3bc297dd28d49fd0dcfa89ab732b14e21dc1057ab23e1ab2b6caa: Status 404 returned error can't find the container with id d079ff1bdae3bc297dd28d49fd0dcfa89ab732b14e21dc1057ab23e1ab2b6caa Oct 29 22:49:55.431091 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:55.431064 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" event={"ID":"2802b231-5ab0-43cd-85f1-c6c4c68c331a","Type":"ContainerStarted","Data":"d079ff1bdae3bc297dd28d49fd0dcfa89ab732b14e21dc1057ab23e1ab2b6caa"} Oct 29 22:49:56.330869 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:56.330825 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:49:57.443475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:57.443436 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" event={"ID":"2802b231-5ab0-43cd-85f1-c6c4c68c331a","Type":"ContainerStarted","Data":"2865294ecd2a0ba5d564c56f3dfab4fb520f074021ba1e3a30c5fdfb71460d4e"} Oct 29 22:49:57.443911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:57.443678 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:49:57.490911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:49:57.490855 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" podStartSLOduration=2.008490073 podStartE2EDuration="3.490825266s" totalImagesPullingTime="1.482335193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:49:57.489276059 +0000 UTC m=+994.601921695" watchObservedRunningTime="2026-10-29 22:49:57.490825266 +0000 UTC m=+994.603470854" Oct 29 22:50:03.454811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:03.454782 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 22:50:04.543550 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.543511 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z"] Oct 29 22:50:04.676678 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.676637 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z"] Oct 29 22:50:04.676881 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.676725 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:04.760443 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.760413 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-944xm\" (UniqueName: \"kubernetes.io/projected/4b14f25f-dafa-4156-aa19-fd2aad88ec80-kube-api-access-944xm\") pod \"llmisvc-controller-manager-6466c554d8-ngv8z\" (UID: \"4b14f25f-dafa-4156-aa19-fd2aad88ec80\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:04.760600 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.760556 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b14f25f-dafa-4156-aa19-fd2aad88ec80-cert\") pod \"llmisvc-controller-manager-6466c554d8-ngv8z\" (UID: \"4b14f25f-dafa-4156-aa19-fd2aad88ec80\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:04.861561 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.861493 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-944xm\" (UniqueName: \"kubernetes.io/projected/4b14f25f-dafa-4156-aa19-fd2aad88ec80-kube-api-access-944xm\") pod \"llmisvc-controller-manager-6466c554d8-ngv8z\" (UID: \"4b14f25f-dafa-4156-aa19-fd2aad88ec80\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:04.861700 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.861608 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b14f25f-dafa-4156-aa19-fd2aad88ec80-cert\") pod \"llmisvc-controller-manager-6466c554d8-ngv8z\" (UID: \"4b14f25f-dafa-4156-aa19-fd2aad88ec80\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:04.864425 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.864396 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b14f25f-dafa-4156-aa19-fd2aad88ec80-cert\") pod \"llmisvc-controller-manager-6466c554d8-ngv8z\" (UID: \"4b14f25f-dafa-4156-aa19-fd2aad88ec80\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:04.883204 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:04.883175 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-944xm\" (UniqueName: \"kubernetes.io/projected/4b14f25f-dafa-4156-aa19-fd2aad88ec80-kube-api-access-944xm\") pod \"llmisvc-controller-manager-6466c554d8-ngv8z\" (UID: \"4b14f25f-dafa-4156-aa19-fd2aad88ec80\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:05.006924 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:05.006886 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:05.331712 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:05.331687 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z"] Oct 29 22:50:05.334361 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:50:05.334334 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b14f25f_dafa_4156_aa19_fd2aad88ec80.slice/crio-46cfe961cc0730587272bbc03bfc41ef22cf1f57e5124af85aaf28b8f52554e5 WatchSource:0}: Error finding container 46cfe961cc0730587272bbc03bfc41ef22cf1f57e5124af85aaf28b8f52554e5: Status 404 returned error can't find the container with id 46cfe961cc0730587272bbc03bfc41ef22cf1f57e5124af85aaf28b8f52554e5 Oct 29 22:50:05.335880 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:05.335860 5030 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 22:50:05.518397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:05.518358 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" event={"ID":"4b14f25f-dafa-4156-aa19-fd2aad88ec80","Type":"ContainerStarted","Data":"46cfe961cc0730587272bbc03bfc41ef22cf1f57e5124af85aaf28b8f52554e5"} Oct 29 22:50:05.794439 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:50:05.794383 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:50:06.525574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.525540 5030 generic.go:356] "Generic (PLEG): container finished" podID="fa079846-a34e-4362-b007-8a85af7a852a" containerID="ede89b4ec41e538d10c89bdc0fab2c2a03fb8cf959e4fbe7acec7a5e73e4690b" exitCode=0 Oct 29 22:50:06.525792 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.525622 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" event={"ID":"fa079846-a34e-4362-b007-8a85af7a852a","Type":"ContainerDied","Data":"ede89b4ec41e538d10c89bdc0fab2c2a03fb8cf959e4fbe7acec7a5e73e4690b"} Oct 29 22:50:06.526105 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.526088 5030 scope.go:122] "RemoveContainer" containerID="ede89b4ec41e538d10c89bdc0fab2c2a03fb8cf959e4fbe7acec7a5e73e4690b" Oct 29 22:50:06.527350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.527321 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:50:06.527449 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.527402 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:50:06.527512 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.527449 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:50:06.527574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.527504 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" event={"ID":"4b14f25f-dafa-4156-aa19-fd2aad88ec80","Type":"ContainerStarted","Data":"70b35d96c0c5941fb12e76bbbd971d29e378ec12887488f22233be88a6968b48"} Oct 29 22:50:06.527574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.527562 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:06.643917 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:06.643833 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" podStartSLOduration=1.9835387629999999 podStartE2EDuration="2.643813395s" totalImagesPullingTime="660.274632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:50:06.641266074 +0000 UTC m=+1003.753911666" watchObservedRunningTime="2026-10-29 22:50:06.643813395 +0000 UTC m=+1003.756458984" Oct 29 22:50:07.535378 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:07.535336 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" event={"ID":"fa079846-a34e-4362-b007-8a85af7a852a","Type":"ContainerStarted","Data":"d8f3b5b033a6247d4f3674393cc168648efe5c379ae2704dd2e7b04a987498db"} Oct 29 22:50:08.330360 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:08.330327 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:50:08.331626 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:08.331597 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 29 22:50:08.331715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:08.331645 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 29 22:50:09.330511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:09.330479 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 29 22:50:09.330933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:09.330531 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 29 22:50:10.330485 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:10.330453 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 29 22:50:10.330649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:10.330502 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 29 22:50:11.330717 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:11.330686 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 29 22:50:11.331134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:11.330737 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 29 22:50:11.902812 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:11.902767 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:50:12.331290 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:12.331256 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 29 22:50:12.331774 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:12.331309 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 29 22:50:13.334337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:13.334313 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:50:13.335326 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:13.335310 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" Oct 29 22:50:35.793781 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:50:35.793699 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:50:37.537428 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.537397 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ngv8z" Oct 29 22:50:37.614943 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.614908 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms"] Oct 29 22:50:37.615246 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.615214 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" podUID="3ad28ae6-cd6c-4fe2-a17a-14c216578d67" containerName="manager" containerID="cri-o://fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947" gracePeriod=30 Oct 29 22:50:37.882980 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.882957 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:50:37.979724 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.979693 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r\" (UniqueName: \"kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r\") pod \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " Oct 29 22:50:37.979939 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.979823 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert\" (UniqueName: \"kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert\") pod \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\" (UID: \"3ad28ae6-cd6c-4fe2-a17a-14c216578d67\") " Oct 29 22:50:37.982061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.982035 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r" pod "3ad28ae6-cd6c-4fe2-a17a-14c216578d67" (UID: "3ad28ae6-cd6c-4fe2-a17a-14c216578d67"). InnerVolumeSpecName "kube-api-access-nfj8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:50:37.982162 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:37.982104 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert" pod "3ad28ae6-cd6c-4fe2-a17a-14c216578d67" (UID: "3ad28ae6-cd6c-4fe2-a17a-14c216578d67"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:50:38.081312 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.081289 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfj8r\" (UniqueName: \"kubernetes.io/projected/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-kube-api-access-nfj8r\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:50:38.081312 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.081313 5030 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ad28ae6-cd6c-4fe2-a17a-14c216578d67-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:50:38.729338 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.729306 5030 generic.go:356] "Generic (PLEG): container finished" podID="3ad28ae6-cd6c-4fe2-a17a-14c216578d67" containerID="fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947" exitCode=0 Oct 29 22:50:38.729760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.729359 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" Oct 29 22:50:38.729760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.729391 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" event={"ID":"3ad28ae6-cd6c-4fe2-a17a-14c216578d67","Type":"ContainerDied","Data":"fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947"} Oct 29 22:50:38.729760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.729449 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms" event={"ID":"3ad28ae6-cd6c-4fe2-a17a-14c216578d67","Type":"ContainerDied","Data":"01978de39dced531126e3fed7f731a740563d7cf79f776a0915be99fbfd81015"} Oct 29 22:50:38.729760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.729479 5030 scope.go:122] "RemoveContainer" containerID="fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947" Oct 29 22:50:38.739519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.739499 5030 scope.go:122] "RemoveContainer" containerID="fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947" Oct 29 22:50:38.739773 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:50:38.739755 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947\": container with ID starting with fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947 not found: ID does not exist" containerID="fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947" Oct 29 22:50:38.739855 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.739782 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947"} err="failed to get container status \"fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947\": rpc error: code = NotFound desc = could not find container \"fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947\": container with ID starting with fcdab88c7790453bb3150f18de148ab757e058a59d9c91c54c746ef6e674b947 not found: ID does not exist" Oct 29 22:50:38.794763 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.794735 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms"] Oct 29 22:50:38.808335 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:38.808307 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tqwms"] Oct 29 22:50:39.044661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:39.044585 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ad28ae6-cd6c-4fe2-a17a-14c216578d67" path="/var/lib/kubelet/pods/3ad28ae6-cd6c-4fe2-a17a-14c216578d67/volumes" Oct 29 22:50:58.172329 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:50:58.172296 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:51:01.105533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:01.105483 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:51:05.831215 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:51:05.831171 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:51:05.852629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:05.852590 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:51:11.136342 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:11.136305 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:51:17.022031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:17.021997 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:51:35.792976 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:51:35.792941 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:51:52.918826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:52.918791 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp"] Oct 29 22:51:52.919601 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:52.919573 5030 cpu_manager.go:414] "containerMap: removing container" podUID="3ad28ae6-cd6c-4fe2-a17a-14c216578d67" containerName="manager" Oct 29 22:51:52.919601 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:52.919597 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ad28ae6-cd6c-4fe2-a17a-14c216578d67" containerName="manager" Oct 29 22:51:52.919878 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:52.919857 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ad28ae6-cd6c-4fe2-a17a-14c216578d67" containerName="manager" Oct 29 22:51:52.924125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:52.924104 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:52.948654 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:52.948625 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp"] Oct 29 22:51:53.062459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062428 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062628 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062468 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062702 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062626 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062702 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062653 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062702 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062678 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062825 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062787 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062825 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062818 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062891 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092a3e1c-2934-486f-8fd9-35cc2e605a01-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.062984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.062916 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brms\" (UniqueName: \"kubernetes.io/projected/092a3e1c-2934-486f-8fd9-35cc2e605a01-kube-api-access-4brms\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164013 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164042 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164045 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164083 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164113 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164157 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092a3e1c-2934-486f-8fd9-35cc2e605a01-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164192 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brms\" (UniqueName: \"kubernetes.io/projected/092a3e1c-2934-486f-8fd9-35cc2e605a01-kube-api-access-4brms\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164381 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164227 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164381 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164250 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164381 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164348 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164527 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164485 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164583 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164543 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164641 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164772 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.164955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.164939 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/092a3e1c-2934-486f-8fd9-35cc2e605a01-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.166696 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.166676 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.166767 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.166711 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.186500 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.186431 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brms\" (UniqueName: \"kubernetes.io/projected/092a3e1c-2934-486f-8fd9-35cc2e605a01-kube-api-access-4brms\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.186644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.186626 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/092a3e1c-2934-486f-8fd9-35cc2e605a01-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-gsfnp\" (UID: \"092a3e1c-2934-486f-8fd9-35cc2e605a01\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.239427 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.239401 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:53.557517 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.557493 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp"] Oct 29 22:51:53.559775 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:51:53.559754 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092a3e1c_2934_486f_8fd9_35cc2e605a01.slice/crio-d484d30020141d0ff3c1dadde5b9df11881fe78575a84811b4dad24aaf1de8c5 WatchSource:0}: Error finding container d484d30020141d0ff3c1dadde5b9df11881fe78575a84811b4dad24aaf1de8c5: Status 404 returned error can't find the container with id d484d30020141d0ff3c1dadde5b9df11881fe78575a84811b4dad24aaf1de8c5 Oct 29 22:51:53.561878 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.561828 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:51:53.561967 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.561913 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:51:53.561967 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:53.561948 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:51:54.195343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:54.195305 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" event={"ID":"092a3e1c-2934-486f-8fd9-35cc2e605a01","Type":"ContainerStarted","Data":"c80fabb6e39022e92e68c93cc7345d68a3d1cbe7e406f2b29feb6e4bfcaa0bb3"} Oct 29 22:51:54.196018 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:54.195361 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" event={"ID":"092a3e1c-2934-486f-8fd9-35cc2e605a01","Type":"ContainerStarted","Data":"d484d30020141d0ff3c1dadde5b9df11881fe78575a84811b4dad24aaf1de8c5"} Oct 29 22:51:54.240115 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:54.240092 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:51:54.241714 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:54.241687 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:51:54.241833 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:54.241729 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:51:55.240689 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:55.240655 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:51:55.241107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:55.240715 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:51:56.240367 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:56.240331 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:51:56.240535 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:56.240381 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:51:57.239993 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:57.239965 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:51:57.240453 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:57.240021 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:51:58.239699 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:58.239669 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:51:58.239906 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:58.239718 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:51:59.240021 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:59.239987 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:51:59.240410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:51:59.240036 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:52:00.240222 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:00.240185 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 22:52:00.240678 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:00.240232 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 22:52:01.806076 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:01.806043 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:52:01.807050 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:01.806988 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:52:01.808392 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:01.808370 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" Oct 29 22:52:01.879629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:01.879582 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podStartSLOduration=9.879568115 podStartE2EDuration="9.879568115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:51:54.275431796 +0000 UTC m=+1111.388077385" watchObservedRunningTime="2026-10-29 22:52:01.879568115 +0000 UTC m=+1118.992213702" Oct 29 22:52:05.793697 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:52:05.793654 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:52:31.664533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.664501 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn"] Oct 29 22:52:31.670252 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.670221 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.691333 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.691306 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-20f88f-serving-cert\"" Oct 29 22:52:31.693936 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.693911 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn"] Oct 29 22:52:31.756985 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.756948 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.757185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.757042 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.757185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.757103 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7rtj\" (UniqueName: \"kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.757185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.757169 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.857911 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.857885 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.858092 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.857993 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.858092 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.858056 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.858185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.858111 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7rtj\" (UniqueName: \"kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.858674 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.858600 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.860552 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.860520 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.860683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.860552 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.880980 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.880959 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7rtj\" (UniqueName: \"kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj\") pod \"maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:31.989082 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:31.989051 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:32.315132 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:32.315106 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn"] Oct 29 22:52:32.317235 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:52:32.317209 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265dfb5f_4ce3_4b9d_8bb4_acdd48bf1273.slice/crio-c1ad2d67b16200c8c5de097832d704ac797381f68a5226a261f66e9e26680942 WatchSource:0}: Error finding container c1ad2d67b16200c8c5de097832d704ac797381f68a5226a261f66e9e26680942: Status 404 returned error can't find the container with id c1ad2d67b16200c8c5de097832d704ac797381f68a5226a261f66e9e26680942 Oct 29 22:52:32.410203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:32.410176 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" event={"ID":"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273","Type":"ContainerStarted","Data":"c1ad2d67b16200c8c5de097832d704ac797381f68a5226a261f66e9e26680942"} Oct 29 22:52:33.417628 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:33.417592 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" event={"ID":"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273","Type":"ContainerStarted","Data":"489b0935b63513783a162a4d9930f6703f8aad0b7bccc0f656b7e5c488dfc427"} Oct 29 22:52:33.418075 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:33.417696 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:33.471167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:33.471110 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" podStartSLOduration=2.16207523 podStartE2EDuration="2.471096488s" totalImagesPullingTime="309.021258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:52:33.469312513 +0000 UTC m=+1150.581958102" watchObservedRunningTime="2026-10-29 22:52:33.471096488 +0000 UTC m=+1150.583742076" Oct 29 22:52:35.263126 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.263096 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw"] Oct 29 22:52:35.267648 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.267626 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.293440 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.293403 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw"] Oct 29 22:52:35.328589 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.328559 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-81f92d-serving-cert\"" Oct 29 22:52:35.401472 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.401441 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g5gq\" (UniqueName: \"kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.401650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.401482 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.401650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.401574 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.401745 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.401710 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.503148 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.503117 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g5gq\" (UniqueName: \"kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.503310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.503157 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.503310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.503198 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.503310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.503282 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.504064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.503990 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.505928 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.505909 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.506006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.505962 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.524760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.524694 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g5gq\" (UniqueName: \"kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq\") pod \"maas-api-e2e-worker-w6-81f92d-6db8456994-627pw\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.585605 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.585576 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:35.616644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.616612 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:52:35.794729 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:52:35.794638 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:52:35.925685 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:35.925634 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw"] Oct 29 22:52:35.928657 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:52:35.928634 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod193b43e0_1296_4296_b7af_001487290791.slice/crio-130ced2277b6a40102999ea50162fe16e04bba50516d52272ecc586e69958f2c WatchSource:0}: Error finding container 130ced2277b6a40102999ea50162fe16e04bba50516d52272ecc586e69958f2c: Status 404 returned error can't find the container with id 130ced2277b6a40102999ea50162fe16e04bba50516d52272ecc586e69958f2c Oct 29 22:52:36.435675 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:36.435647 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" event={"ID":"193b43e0-1296-4296-b7af-001487290791","Type":"ContainerStarted","Data":"0ba12be35e2aa4b3569d25e2e3023feb631546402bbb3504868f8fcfce43e439"} Oct 29 22:52:36.436140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:36.435696 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" event={"ID":"193b43e0-1296-4296-b7af-001487290791","Type":"ContainerStarted","Data":"130ced2277b6a40102999ea50162fe16e04bba50516d52272ecc586e69958f2c"} Oct 29 22:52:36.436140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:36.435757 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:36.481903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:36.481848 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" podStartSLOduration=1.150227746 podStartE2EDuration="1.4818214s" totalImagesPullingTime="331.593654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:52:36.480441907 +0000 UTC m=+1153.593087494" watchObservedRunningTime="2026-10-29 22:52:36.4818214 +0000 UTC m=+1153.594466988" Oct 29 22:52:37.134457 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.134429 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s"] Oct 29 22:52:37.138807 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.138787 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.157308 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.157277 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s"] Oct 29 22:52:37.185938 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.185901 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-dd3ff3-serving-cert\"" Oct 29 22:52:37.321285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.321247 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.321462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.321326 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.321515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.321460 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.321515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.321492 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s57s\" (UniqueName: \"kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.423276 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.423194 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.423276 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.423250 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.423528 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.423315 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.423528 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.423425 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s57s\" (UniqueName: \"kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.424300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.424275 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.425890 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.425861 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.425890 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.425867 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.445035 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.445013 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s57s\" (UniqueName: \"kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s\") pod \"maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.452827 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.452807 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:37.759596 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:37.759568 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s"] Oct 29 22:52:37.761738 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:52:37.761706 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5048ca2_d798_4454_aa96_a4e0f180dd1d.slice/crio-6ed48e177ac5d882c1c9fb98848e23c3307606f36128cfebad0d5532849c8204 WatchSource:0}: Error finding container 6ed48e177ac5d882c1c9fb98848e23c3307606f36128cfebad0d5532849c8204: Status 404 returned error can't find the container with id 6ed48e177ac5d882c1c9fb98848e23c3307606f36128cfebad0d5532849c8204 Oct 29 22:52:38.449231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.449149 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" event={"ID":"d5048ca2-d798-4454-aa96-a4e0f180dd1d","Type":"ContainerStarted","Data":"0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c"} Oct 29 22:52:38.449231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.449213 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" event={"ID":"d5048ca2-d798-4454-aa96-a4e0f180dd1d","Type":"ContainerStarted","Data":"6ed48e177ac5d882c1c9fb98848e23c3307606f36128cfebad0d5532849c8204"} Oct 29 22:52:38.449669 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.449245 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:38.494669 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.494620 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" podStartSLOduration=1.114276847 podStartE2EDuration="1.494607462s" totalImagesPullingTime="380.330615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:52:38.492593413 +0000 UTC m=+1155.605239001" watchObservedRunningTime="2026-10-29 22:52:38.494607462 +0000 UTC m=+1155.607253044" Oct 29 22:52:38.765397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.765366 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp"] Oct 29 22:52:38.769583 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.769561 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:38.788181 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.788153 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-a2e7c8-serving-cert\"" Oct 29 22:52:38.790729 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.790478 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp"] Oct 29 22:52:38.945018 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.944981 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:38.945221 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.945056 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:38.945221 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.945180 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k45f4\" (UniqueName: \"kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:38.945358 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:38.945276 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.045961 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.045886 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.045961 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.045956 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.046175 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.046011 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.046175 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.046098 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k45f4\" (UniqueName: \"kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.046697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.046625 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.048544 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.048523 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.048692 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.048670 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.067129 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.067109 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k45f4\" (UniqueName: \"kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4\") pod \"maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.083919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.083893 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:39.387748 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.387727 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp"] Oct 29 22:52:39.389295 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:52:39.389259 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e10dc5_da26_495c_8e26_574a86c666b0.slice/crio-e2303a0576efcc287b234e89c7ba926d8f64069b0aea0cf236abd48ad10f10b8 WatchSource:0}: Error finding container e2303a0576efcc287b234e89c7ba926d8f64069b0aea0cf236abd48ad10f10b8: Status 404 returned error can't find the container with id e2303a0576efcc287b234e89c7ba926d8f64069b0aea0cf236abd48ad10f10b8 Oct 29 22:52:39.428506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.428480 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 22:52:39.456398 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:39.456367 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" event={"ID":"51e10dc5-da26-495c-8e26-574a86c666b0","Type":"ContainerStarted","Data":"e2303a0576efcc287b234e89c7ba926d8f64069b0aea0cf236abd48ad10f10b8"} Oct 29 22:52:40.234069 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.234037 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:52:40.464155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.464121 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" event={"ID":"51e10dc5-da26-495c-8e26-574a86c666b0","Type":"ContainerStarted","Data":"b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f"} Oct 29 22:52:40.464525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.464250 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:40.559985 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.559917 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc"] Oct 29 22:52:40.564479 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.564455 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.572098 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.572054 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" podStartSLOduration=2.139645731 podStartE2EDuration="2.572041195s" totalImagesPullingTime="432.395464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:52:40.564117502 +0000 UTC m=+1157.676763093" watchObservedRunningTime="2026-10-29 22:52:40.572041195 +0000 UTC m=+1157.684686783" Oct 29 22:52:40.592332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.592305 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-07eef2-serving-cert\"" Oct 29 22:52:40.600758 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.600736 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc"] Oct 29 22:52:40.665228 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.665201 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.665418 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.665248 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.665418 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.665378 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8pfv\" (UniqueName: \"kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.665532 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.665438 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.766986 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.766956 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.767178 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.767024 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.767178 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.767067 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8pfv\" (UniqueName: \"kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.767178 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.767117 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.767692 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.767667 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.769550 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.769531 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.769619 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.769568 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.788833 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.788793 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8pfv\" (UniqueName: \"kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv\") pod \"maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:40.882641 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:40.882558 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:41.200015 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:41.199981 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc"] Oct 29 22:52:41.202004 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:52:41.201978 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd944d8e4_7578_43f0_a586_c61f10d4f8f0.slice/crio-2515fed92eb9362431b431f4dafdcfe9ec989a9a459d2666681709209c8fd6eb WatchSource:0}: Error finding container 2515fed92eb9362431b431f4dafdcfe9ec989a9a459d2666681709209c8fd6eb: Status 404 returned error can't find the container with id 2515fed92eb9362431b431f4dafdcfe9ec989a9a459d2666681709209c8fd6eb Oct 29 22:52:41.476711 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:41.476670 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" event={"ID":"d944d8e4-7578-43f0-a586-c61f10d4f8f0","Type":"ContainerStarted","Data":"2515fed92eb9362431b431f4dafdcfe9ec989a9a459d2666681709209c8fd6eb"} Oct 29 22:52:42.447999 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:42.447975 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 22:52:42.485899 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:42.485870 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" event={"ID":"d944d8e4-7578-43f0-a586-c61f10d4f8f0","Type":"ContainerStarted","Data":"072c0048248906662b0d96702d9795032562fd8c5a369841f6a8c74c84b73f1c"} Oct 29 22:52:42.486324 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:42.486006 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:44.462471 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:44.462437 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:52:44.509965 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:44.509921 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" podStartSLOduration=4.21109068 podStartE2EDuration="4.50990755s" totalImagesPullingTime="298.81687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:52:42.546882027 +0000 UTC m=+1159.659527615" watchObservedRunningTime="2026-10-29 22:52:44.50990755 +0000 UTC m=+1161.622553145" Oct 29 22:52:45.877472 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:45.877438 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:52:46.482453 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:46.482420 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 22:52:48.498620 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:48.498591 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 22:52:50.969433 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:50.969390 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:52:59.248749 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:52:59.248719 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:53:05.795162 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:53:05.795123 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:53:08.334350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:08.334309 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:53:20.208008 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:20.207968 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:53:23.190621 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:23.190592 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:53:23.194974 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:23.194952 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:53:32.432142 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:32.432107 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:53:35.795753 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:53:35.795673 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:53:43.814535 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:43.814498 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:53:50.231008 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.230975 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2"] Oct 29 22:53:50.235778 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.235752 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.255529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.255478 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.255801 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.255763 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnsbc\" (UniqueName: \"kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.256978 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.255887 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.256978 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.255939 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.256978 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.255986 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.256978 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.256092 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.272797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.271828 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-5bf55a\"/\"default-dockercfg-4nfhm\"" Oct 29 22:53:50.272797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.271975 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-5bf55a\"/\"premium-w5-5bf55a-kserve-self-signed-certs\"" Oct 29 22:53:50.275312 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.275287 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2"] Oct 29 22:53:50.277410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.277389 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-5bf55a\"/\"kube-root-ca.crt\"" Oct 29 22:53:50.277521 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.277460 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-5bf55a\"/\"openshift-service-ca.crt\"" Oct 29 22:53:50.357119 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357087 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357287 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357156 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnsbc\" (UniqueName: \"kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357287 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357219 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357287 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357261 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357458 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357308 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357458 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357410 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357620 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357591 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357714 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357634 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.357796 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.357779 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.359809 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.359785 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.360123 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.360104 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.379503 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.379477 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnsbc\" (UniqueName: \"kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc\") pod \"premium-w5-5bf55a-kserve-7fd9795857-q8xp2\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.551583 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.551512 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:53:50.860305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.860271 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2"] Oct 29 22:53:50.863046 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:53:50.863016 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0661d7c_66fc_4a2b_8251_60f6b7bc0c82.slice/crio-4e93bb319bff1f98b82627ba22fc398f596a1fbe6a8359cf42f08985b4363a89 WatchSource:0}: Error finding container 4e93bb319bff1f98b82627ba22fc398f596a1fbe6a8359cf42f08985b4363a89: Status 404 returned error can't find the container with id 4e93bb319bff1f98b82627ba22fc398f596a1fbe6a8359cf42f08985b4363a89 Oct 29 22:53:50.942211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:50.942178 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" event={"ID":"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82","Type":"ContainerStarted","Data":"4e93bb319bff1f98b82627ba22fc398f596a1fbe6a8359cf42f08985b4363a89"} Oct 29 22:53:52.957212 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:52.957168 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" event={"ID":"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82","Type":"ContainerStarted","Data":"a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb"} Oct 29 22:53:53.003114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:53.003066 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podStartSLOduration=1.8942552419999998 podStartE2EDuration="3.003052288s" totalImagesPullingTime="1.108797046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:53:53.000378176 +0000 UTC m=+1230.113023766" watchObservedRunningTime="2026-10-29 22:53:53.003052288 +0000 UTC m=+1230.115697906" Oct 29 22:53:53.552012 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:53:53.551979 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:54:00.552246 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:00.552214 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:54:00.566091 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:00.566069 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:54:01.030534 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:01.030504 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:54:05.636212 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:05.636180 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:54:05.800529 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:54:05.800490 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:54:10.908791 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:10.908763 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5"] Oct 29 22:54:10.913044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:10.913025 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:10.950241 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:10.950209 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5"] Oct 29 22:54:11.011774 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.011743 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.011977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.011799 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.011977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.011902 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.011977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.011932 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.011977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.011958 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.012125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.012025 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.012125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.012053 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.012187 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.012131 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6nxc\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.012187 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.012159 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.025678 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.025657 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-dd3ff3-openshift-default-dockercfg-528h2\"" Oct 29 22:54:11.112760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.112727 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.112955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.112802 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.112955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.112863 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.112955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.112897 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.112955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.112928 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113169 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.112964 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113169 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113020 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113169 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113043 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113169 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113105 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6nxc\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113398 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113313 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113398 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113389 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113558 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113656 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113586 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.113656 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.113600 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.115432 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.115405 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.116306 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.116279 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.140414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.140383 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.140692 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.140670 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6nxc\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.230408 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.230369 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:11.554695 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.554608 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5"] Oct 29 22:54:11.560264 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:54:11.559535 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a54399_f2a4_4e8d_86e6_c8acc9b4f90b.slice/crio-6c2dfaa13a11be029c7dafc4d881d2347be0bccfe2ac0384e11f0829300e052c WatchSource:0}: Error finding container 6c2dfaa13a11be029c7dafc4d881d2347be0bccfe2ac0384e11f0829300e052c: Status 404 returned error can't find the container with id 6c2dfaa13a11be029c7dafc4d881d2347be0bccfe2ac0384e11f0829300e052c Oct 29 22:54:11.562332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.562294 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:11.562420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.562368 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:11.562420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:11.562400 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:12.091252 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:12.091216 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" event={"ID":"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b","Type":"ContainerStarted","Data":"ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020"} Oct 29 22:54:12.091735 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:12.091277 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" event={"ID":"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b","Type":"ContainerStarted","Data":"6c2dfaa13a11be029c7dafc4d881d2347be0bccfe2ac0384e11f0829300e052c"} Oct 29 22:54:12.157384 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:12.157138 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podStartSLOduration=2.157120075 podStartE2EDuration="2.157120075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:54:12.153960462 +0000 UTC m=+1249.266606055" watchObservedRunningTime="2026-10-29 22:54:12.157120075 +0000 UTC m=+1249.269765665" Oct 29 22:54:12.230965 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:12.230934 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:12.232323 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:12.232295 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:12.232434 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:12.232346 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:13.169862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.169116 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9"] Oct 29 22:54:13.193481 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.193456 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.202488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.202462 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9"] Oct 29 22:54:13.227215 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.227191 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ffba57\"/\"openshift-service-ca.crt\"" Oct 29 22:54:13.227399 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.227203 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ffba57\"/\"kube-root-ca.crt\"" Oct 29 22:54:13.227488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.227317 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ffba57\"/\"premium-w0-ffba57-kserve-self-signed-certs\"" Oct 29 22:54:13.227488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.227417 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ffba57\"/\"default-dockercfg-h5mgq\"" Oct 29 22:54:13.231576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.231544 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:13.231706 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.231602 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:13.233229 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.233206 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.233475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.233442 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fntcf\" (UniqueName: \"kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.233580 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.233525 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.233580 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.233557 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.233730 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.233621 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.233811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.233731 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.334528 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.334488 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.334724 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.334579 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.334724 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.334632 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fntcf\" (UniqueName: \"kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.334724 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.334702 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.334925 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.334731 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.334925 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.334801 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.335240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.335171 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.335806 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.335786 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.336001 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.335876 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.337827 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.337807 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.337943 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.337808 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.360612 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.360557 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fntcf\" (UniqueName: \"kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf\") pod \"premium-w0-ffba57-kserve-bdd764b6f-9xss9\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.513464 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.513438 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:13.858854 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:13.858754 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9"] Oct 29 22:54:13.887863 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:54:13.884346 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod240b0a83_97d1_466a_a216_3ccbb0180f5c.slice/crio-ec362a9ce98b715401f6cccf7c3944ae81095986650ed0471dfe6f9f42661bf3 WatchSource:0}: Error finding container ec362a9ce98b715401f6cccf7c3944ae81095986650ed0471dfe6f9f42661bf3: Status 404 returned error can't find the container with id ec362a9ce98b715401f6cccf7c3944ae81095986650ed0471dfe6f9f42661bf3 Oct 29 22:54:14.105343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:14.105301 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" event={"ID":"240b0a83-97d1-466a-a216-3ccbb0180f5c","Type":"ContainerStarted","Data":"81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126"} Oct 29 22:54:14.105541 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:14.105359 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" event={"ID":"240b0a83-97d1-466a-a216-3ccbb0180f5c","Type":"ContainerStarted","Data":"ec362a9ce98b715401f6cccf7c3944ae81095986650ed0471dfe6f9f42661bf3"} Oct 29 22:54:14.169912 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:14.168503 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podStartSLOduration=1.168483902 podStartE2EDuration="1.168483902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:54:14.16657666 +0000 UTC m=+1251.279222251" watchObservedRunningTime="2026-10-29 22:54:14.168483902 +0000 UTC m=+1251.281129492" Oct 29 22:54:14.230907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:14.230832 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:14.231071 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:14.230925 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:14.513597 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:14.513568 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:15.231462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:15.231426 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:15.231966 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:15.231490 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:16.231190 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:16.231158 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:16.231368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:16.231221 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:17.231003 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:17.230957 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:17.231489 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:17.231023 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:18.231453 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:18.231408 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 22:54:18.232004 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:18.231459 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 22:54:19.450996 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:19.450956 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:19.451581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:19.451496 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:19.452410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:19.452389 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:54:23.514207 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:23.514170 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:23.530255 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:23.530231 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:24.194771 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:24.194743 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:54:29.257310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:29.257258 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:54:35.796393 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:54:35.796358 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:54:55.758445 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.758405 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9"] Oct 29 22:54:55.764598 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.764574 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.785600 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.785576 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-5bf55a-openshift-default-dockercfg-hx7k2\"" Oct 29 22:54:55.803230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.803196 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9"] Oct 29 22:54:55.831445 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831412 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.831645 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831463 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.831645 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831494 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.832033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831644 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.832033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831668 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.832033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831823 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.832033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831911 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.832033 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.831936 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wwvg\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.832273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.832059 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.922395 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.922366 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc"] Oct 29 22:54:55.926597 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.926577 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:55.932970 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.932912 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933180 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933215 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wwvg\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933261 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933436 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933313 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933493 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933425 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933502 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933549 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933538 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933589 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933619 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933888 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.933941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.933928 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.934275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.934068 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.934275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.934225 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.936094 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.936074 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.936420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.936404 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.948730 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.948711 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc"] Oct 29 22:54:55.963794 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.963767 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wwvg\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.971383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.971356 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token\") pod \"e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:55.983919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:55.983898 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-07eef2-openshift-default-dockercfg-5l9dg\"" Oct 29 22:54:56.034541 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034483 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.034648 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034541 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.034648 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034637 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.034757 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034677 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.034866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034816 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.034994 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034915 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.035060 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.034997 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.035123 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.035088 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.035168 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.035122 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt79c\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.079724 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.079695 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:56.137240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137206 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137308 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137386 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137462 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137416 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt79c\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137583 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137470 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137581 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137713 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137654 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.137713 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137705 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.139459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.137832 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.139459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.138639 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.139459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.138830 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.139459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.139338 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.139459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.139378 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.139808 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.139491 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.140740 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.140720 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.140886 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.140755 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.166877 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.165617 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.167642 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.167575 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt79c\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.306499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.306419 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:56.403114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.403089 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9"] Oct 29 22:54:56.405693 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:54:56.405658 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344cdc2c_85cc_4eb5_bb00_523b63b26b2c.slice/crio-ac994be78d9a2646e86425ebe20991b310da3d52d486a78518fb6e7646a0a655 WatchSource:0}: Error finding container ac994be78d9a2646e86425ebe20991b310da3d52d486a78518fb6e7646a0a655: Status 404 returned error can't find the container with id ac994be78d9a2646e86425ebe20991b310da3d52d486a78518fb6e7646a0a655 Oct 29 22:54:56.409289 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.409241 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:56.409390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.409371 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:56.409435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.409422 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:56.646456 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.646428 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc"] Oct 29 22:54:56.647890 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:54:56.647860 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8abd35f7_2058_48c1_bdff_25ed40018ef8.slice/crio-0c4daf2a0559913ed4b07632d47f4b5c6e7305b99d2a728174fe1e5fb282d070 WatchSource:0}: Error finding container 0c4daf2a0559913ed4b07632d47f4b5c6e7305b99d2a728174fe1e5fb282d070: Status 404 returned error can't find the container with id 0c4daf2a0559913ed4b07632d47f4b5c6e7305b99d2a728174fe1e5fb282d070 Oct 29 22:54:56.650712 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.650670 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:56.650886 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.650756 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:56.650886 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:56.650796 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:54:57.413860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:57.411748 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" event={"ID":"344cdc2c-85cc-4eb5-bb00-523b63b26b2c","Type":"ContainerStarted","Data":"43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9"} Oct 29 22:54:57.413860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:57.411809 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" event={"ID":"344cdc2c-85cc-4eb5-bb00-523b63b26b2c","Type":"ContainerStarted","Data":"ac994be78d9a2646e86425ebe20991b310da3d52d486a78518fb6e7646a0a655"} Oct 29 22:54:57.413860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:57.413673 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" event={"ID":"8abd35f7-2058-48c1-bdff-25ed40018ef8","Type":"ContainerStarted","Data":"531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed"} Oct 29 22:54:57.413860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:57.413721 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" event={"ID":"8abd35f7-2058-48c1-bdff-25ed40018ef8","Type":"ContainerStarted","Data":"0c4daf2a0559913ed4b07632d47f4b5c6e7305b99d2a728174fe1e5fb282d070"} Oct 29 22:54:57.492862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:57.485908 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podStartSLOduration=2.4858877059999998 podStartE2EDuration="2.485887706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:54:57.485358147 +0000 UTC m=+1294.598003751" watchObservedRunningTime="2026-10-29 22:54:57.485887706 +0000 UTC m=+1294.598533297" Oct 29 22:54:57.539176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:57.539107 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podStartSLOduration=2.539090615 podStartE2EDuration="2.539090615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:54:57.537518367 +0000 UTC m=+1294.650163993" watchObservedRunningTime="2026-10-29 22:54:57.539090615 +0000 UTC m=+1294.651736203" Oct 29 22:54:58.082212 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.082183 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:54:58.092009 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.091979 5030 patch_prober.go:29] interesting pod/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 22:54:58.092330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.092299 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 22:54:58.306999 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.306971 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:54:58.308815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.308788 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 22:54:58.308951 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.308889 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 22:54:58.502688 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:58.498491 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:54:59.085864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:59.084828 5030 patch_prober.go:29] interesting pod/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 22:54:59.085864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:59.084947 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 22:54:59.313864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:59.310207 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 22:54:59.313864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:54:59.310265 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 22:55:00.082214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:00.082177 5030 patch_prober.go:29] interesting pod/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 22:55:00.083297 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:00.082814 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 22:55:00.317687 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:00.317209 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 22:55:00.317687 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:00.317296 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 22:55:01.083241 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:01.083206 5030 patch_prober.go:29] interesting pod/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 22:55:01.083877 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:01.083810 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 22:55:01.317114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:01.317078 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 22:55:01.317409 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:01.317377 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 22:55:02.080242 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:02.080208 5030 patch_prober.go:29] interesting pod/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 22:55:02.080570 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:02.080515 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 22:55:02.356922 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:02.341811 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 22:55:02.356922 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:02.345917 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 22:55:02.887860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:02.887179 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:55:02.887860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:02.887260 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-gsfnp" podUID="092a3e1c-2934-486f-8fd9-35cc2e605a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:55:03.081873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:03.081003 5030 patch_prober.go:29] interesting pod/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 22:55:03.081873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:03.081059 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 22:55:03.307422 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:03.307381 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 22:55:03.307598 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:03.307451 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 22:55:04.085582 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:04.085548 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:55:04.086109 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:04.085970 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:55:04.086874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:04.086824 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 22:55:04.665907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:04.665514 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:55:04.666929 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:04.666908 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:55:04.670446 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:04.670422 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 22:55:05.796366 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:55:05.796333 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:55:08.411925 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.411887 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv"] Oct 29 22:55:08.416211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.416191 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.456303 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.456275 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv"] Oct 29 22:55:08.589329 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589292 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589503 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589346 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589503 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589436 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589582 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589543 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589613 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6326e02-98c1-4554-8590-11a71357b8c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589697 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589677 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6326e02-98c1-4554-8590-11a71357b8c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589749 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6326e02-98c1-4554-8590-11a71357b8c7-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589789 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf772\" (UniqueName: \"kubernetes.io/projected/b6326e02-98c1-4554-8590-11a71357b8c7-kube-api-access-pf772\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.589948 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.589929 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691038 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.690945 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691067 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691089 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691105 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691149 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691299 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6326e02-98c1-4554-8590-11a71357b8c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691352 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6326e02-98c1-4554-8590-11a71357b8c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691368 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691424 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6326e02-98c1-4554-8590-11a71357b8c7-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691658 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691480 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf772\" (UniqueName: \"kubernetes.io/projected/b6326e02-98c1-4554-8590-11a71357b8c7-kube-api-access-pf772\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691658 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691487 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691658 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691528 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.691658 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.691573 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.692243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.692216 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6326e02-98c1-4554-8590-11a71357b8c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.693627 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.693607 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6326e02-98c1-4554-8590-11a71357b8c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.693956 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.693936 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6326e02-98c1-4554-8590-11a71357b8c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.714468 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.714445 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6326e02-98c1-4554-8590-11a71357b8c7-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.714604 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.714583 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf772\" (UniqueName: \"kubernetes.io/projected/b6326e02-98c1-4554-8590-11a71357b8c7-kube-api-access-pf772\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-8m9xv\" (UID: \"b6326e02-98c1-4554-8590-11a71357b8c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:08.730473 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:08.730455 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:09.062734 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:09.062705 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6326e02_98c1_4554_8590_11a71357b8c7.slice/crio-1d5ca38b7cd4bd6237b1cbe787b00ee3eee1c556f1fe08474113cf6d3a4a28bc WatchSource:0}: Error finding container 1d5ca38b7cd4bd6237b1cbe787b00ee3eee1c556f1fe08474113cf6d3a4a28bc: Status 404 returned error can't find the container with id 1d5ca38b7cd4bd6237b1cbe787b00ee3eee1c556f1fe08474113cf6d3a4a28bc Oct 29 22:55:09.063752 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.063725 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv"] Oct 29 22:55:09.065264 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.065231 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:09.065355 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.065303 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:09.065355 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.065332 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:09.624777 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.624738 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" event={"ID":"b6326e02-98c1-4554-8590-11a71357b8c7","Type":"ContainerStarted","Data":"8ba0e2d40d77e25caf0d2c4da9769c22f11a2305192817df747bae294ea02e40"} Oct 29 22:55:09.625225 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.624796 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" event={"ID":"b6326e02-98c1-4554-8590-11a71357b8c7","Type":"ContainerStarted","Data":"1d5ca38b7cd4bd6237b1cbe787b00ee3eee1c556f1fe08474113cf6d3a4a28bc"} Oct 29 22:55:09.702427 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.702366 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podStartSLOduration=1.702344955 podStartE2EDuration="1.702344955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:09.686453137 +0000 UTC m=+1306.799098753" watchObservedRunningTime="2026-10-29 22:55:09.702344955 +0000 UTC m=+1306.814990543" Oct 29 22:55:09.730870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:09.730818 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:10.738165 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:10.738131 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:10.738669 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:10.738178 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:11.120767 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.120671 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv"] Oct 29 22:55:11.125506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.125480 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.147321 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.147299 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv"] Oct 29 22:55:11.225988 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.225958 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226002 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226086 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmqfk\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226131 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226353 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226274 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226353 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226322 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226446 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226406 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226494 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226449 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.226593 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.226569 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.327566 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327531 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.327566 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327574 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.327819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327590 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.327819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327611 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.327819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327745 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328017 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327943 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328017 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.327984 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328024 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmqfk\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328055 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328083 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328125 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328053 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328170 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328251 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.328468 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.328443 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.330243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.330221 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.330498 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.330482 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.369861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.369399 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmqfk\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.387904 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.376330 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.445429 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.445399 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:11.731441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.731404 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:11.731634 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.731462 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:11.816078 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.812688 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv"] Oct 29 22:55:11.821477 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:11.821450 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f977dc_3582_4a2d_b706_4dfd573a4863.slice/crio-ed496f9b4bfee76b0af85bf6d1025331fd9f65453814ac3cf04caf70456d339a WatchSource:0}: Error finding container ed496f9b4bfee76b0af85bf6d1025331fd9f65453814ac3cf04caf70456d339a: Status 404 returned error can't find the container with id ed496f9b4bfee76b0af85bf6d1025331fd9f65453814ac3cf04caf70456d339a Oct 29 22:55:11.824209 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.824176 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:11.824275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.824241 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:11.824275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:11.824269 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:12.656654 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:12.656612 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" event={"ID":"34f977dc-3582-4a2d-b706-4dfd573a4863","Type":"ContainerStarted","Data":"36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668"} Oct 29 22:55:12.656964 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:12.656940 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" event={"ID":"34f977dc-3582-4a2d-b706-4dfd573a4863","Type":"ContainerStarted","Data":"ed496f9b4bfee76b0af85bf6d1025331fd9f65453814ac3cf04caf70456d339a"} Oct 29 22:55:12.732405 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:12.732366 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:12.732584 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:12.732434 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:12.732584 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:12.732501 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podStartSLOduration=1.732482156 podStartE2EDuration="1.732482156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:12.732104652 +0000 UTC m=+1309.844750240" watchObservedRunningTime="2026-10-29 22:55:12.732482156 +0000 UTC m=+1309.845127743" Oct 29 22:55:13.445734 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:13.445708 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:13.447644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:13.447618 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 22:55:13.447799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:13.447781 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 22:55:13.732242 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:13.732209 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:13.732505 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:13.732481 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:14.445832 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:14.445797 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 22:55:14.446230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:14.445890 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 22:55:14.730867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:14.730824 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:14.731041 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:14.730887 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:15.445824 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:15.445785 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 22:55:15.446016 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:15.445873 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 22:55:15.731382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:15.731350 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:15.731588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:15.731413 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:16.446274 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:16.446234 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 22:55:16.446830 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:16.446301 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 22:55:16.731505 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:16.731465 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 29 22:55:16.731727 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:16.731535 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 29 22:55:17.446262 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:17.446217 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 22:55:17.446466 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:17.446284 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 22:55:18.316214 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:18.316180 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:18.316663 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:18.316596 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:18.317892 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:18.317832 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" Oct 29 22:55:18.445814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:18.445777 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 22:55:18.446085 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:18.445850 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 22:55:19.491525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:19.491492 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:19.492065 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:19.491813 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:19.492900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:19.492876 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:55:24.563811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:24.563772 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:55:26.621713 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.621680 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz"] Oct 29 22:55:26.627324 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.627303 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.647804 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.647774 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz"] Oct 29 22:55:26.658140 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.658112 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ffba57\"/\"distinct-w0-ffba57-kserve-self-signed-certs\"" Oct 29 22:55:26.726237 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.726204 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.726430 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.726365 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.726430 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.726406 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.726533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.726507 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.726592 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.726574 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.726635 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.726621 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svtnw\" (UniqueName: \"kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.827948 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.827913 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svtnw\" (UniqueName: \"kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.828152 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.828054 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.828226 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.828161 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.828226 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.828192 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.828336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.828234 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.828336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.828319 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.828900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.828824 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.830311 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.829193 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.830311 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.829732 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.832788 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.832766 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.834484 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.834462 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.863215 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.851804 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svtnw\" (UniqueName: \"kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw\") pod \"distinct-w0-ffba57-kserve-56698887b8-557nz\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:26.948791 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:26.948288 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:27.275478 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:27.274956 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz"] Oct 29 22:55:27.283054 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:27.283010 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf68acac_097f_49e9_b78c_57b2b90b64d1.slice/crio-f97e2b988d5830566b709951c3cf4f76ed35c44c04ce0a50f57cd6cb28c3c1a8 WatchSource:0}: Error finding container f97e2b988d5830566b709951c3cf4f76ed35c44c04ce0a50f57cd6cb28c3c1a8: Status 404 returned error can't find the container with id f97e2b988d5830566b709951c3cf4f76ed35c44c04ce0a50f57cd6cb28c3c1a8 Oct 29 22:55:27.840066 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:27.840027 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" event={"ID":"cf68acac-097f-49e9-b78c-57b2b90b64d1","Type":"ContainerStarted","Data":"818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0"} Oct 29 22:55:27.840440 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:27.840080 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" event={"ID":"cf68acac-097f-49e9-b78c-57b2b90b64d1","Type":"ContainerStarted","Data":"f97e2b988d5830566b709951c3cf4f76ed35c44c04ce0a50f57cd6cb28c3c1a8"} Oct 29 22:55:27.884457 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:27.884404 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podStartSLOduration=1.8843910529999999 podStartE2EDuration="1.884391053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:27.883573686 +0000 UTC m=+1324.996219271" watchObservedRunningTime="2026-10-29 22:55:27.884391053 +0000 UTC m=+1324.997036640" Oct 29 22:55:27.948887 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:27.948856 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:35.796421 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:55:35.796383 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:55:36.948689 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:36.948656 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:36.967826 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:36.967803 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:37.927523 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:37.927495 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:55:40.160663 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.160629 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m"] Oct 29 22:55:40.165254 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.165231 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.196994 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.196965 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-20f88f-openshift-default-dockercfg-7q76j\"" Oct 29 22:55:40.201906 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.201877 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m"] Oct 29 22:55:40.301640 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.301601 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.301830 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.301650 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.301830 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.301756 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ksx5\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.301975 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.301855 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.301975 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.301910 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.302061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.301979 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.302061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.302034 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.302182 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.302161 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.302264 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.302250 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403446 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403412 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403631 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403454 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403631 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403479 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ksx5\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403631 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403528 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403631 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403584 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403631 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403606 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403918 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403647 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403918 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403736 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.403984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.403946 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.404048 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.404020 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.404121 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.404099 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.404172 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.404120 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.404226 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.404214 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.404955 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.404931 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.406181 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.406154 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.406537 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.406518 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.433813 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.433754 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.433943 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.433886 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ksx5\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.481518 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.481464 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:40.809206 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.809174 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m"] Oct 29 22:55:40.809662 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:40.809631 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202a1cb6_c685_4178_b908_6a53a7818269.slice/crio-e1b531f664a778b1de23eb25e5c6982fcd0c71391d8d726a7e93b22d647566ac WatchSource:0}: Error finding container e1b531f664a778b1de23eb25e5c6982fcd0c71391d8d726a7e93b22d647566ac: Status 404 returned error can't find the container with id e1b531f664a778b1de23eb25e5c6982fcd0c71391d8d726a7e93b22d647566ac Oct 29 22:55:40.812056 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.812020 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:40.812141 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.812089 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:40.812141 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.812117 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:40.950688 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.950646 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" event={"ID":"202a1cb6-c685-4178-b908-6a53a7818269","Type":"ContainerStarted","Data":"45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000"} Oct 29 22:55:40.950929 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:40.950710 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" event={"ID":"202a1cb6-c685-4178-b908-6a53a7818269","Type":"ContainerStarted","Data":"e1b531f664a778b1de23eb25e5c6982fcd0c71391d8d726a7e93b22d647566ac"} Oct 29 22:55:41.003418 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.003359 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podStartSLOduration=1.003340197 podStartE2EDuration="1.003340197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:40.998738555 +0000 UTC m=+1338.111384143" watchObservedRunningTime="2026-10-29 22:55:41.003340197 +0000 UTC m=+1338.115985789" Oct 29 22:55:41.079337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.079240 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs"] Oct 29 22:55:41.085107 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.085080 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.112811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.112765 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs"] Oct 29 22:55:41.116745 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.116707 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.116910 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.116752 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.116910 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.116781 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.116910 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.116902 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.117090 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.116978 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8gvn\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.117090 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.117016 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.117193 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.117092 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.117193 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.117180 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.117289 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.117202 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218086 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218052 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218086 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218085 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218191 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218216 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218244 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218316 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218377 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gvn\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218422 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218489 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.218574 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218504 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.219025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218780 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.219025 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.218818 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.219221 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.219197 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.219457 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.219412 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.221096 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.221072 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.221744 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.221724 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.244273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.244247 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.244591 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.244571 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gvn\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.415914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.414315 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:41.495863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.494977 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:41.495863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.495116 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:41.495863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.495195 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:41.806320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.806285 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs"] Oct 29 22:55:41.807617 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:41.807592 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a3a662f_78c1_486f_b827_fe223058403d.slice/crio-6b2f5c57cbe020bc414bc8970bed2f807fdc48bd210318c5970cf9e9905b37f7 WatchSource:0}: Error finding container 6b2f5c57cbe020bc414bc8970bed2f807fdc48bd210318c5970cf9e9905b37f7: Status 404 returned error can't find the container with id 6b2f5c57cbe020bc414bc8970bed2f807fdc48bd210318c5970cf9e9905b37f7 Oct 29 22:55:41.810344 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.810312 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:41.810441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.810374 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:41.810441 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.810401 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:41.998489 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.998227 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" event={"ID":"0a3a662f-78c1-486f-b827-fe223058403d","Type":"ContainerStarted","Data":"1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969"} Oct 29 22:55:41.998489 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:41.998289 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" event={"ID":"0a3a662f-78c1-486f-b827-fe223058403d","Type":"ContainerStarted","Data":"6b2f5c57cbe020bc414bc8970bed2f807fdc48bd210318c5970cf9e9905b37f7"} Oct 29 22:55:42.062918 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:42.049736 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podStartSLOduration=1.049717094 podStartE2EDuration="1.049717094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:42.04838399 +0000 UTC m=+1339.161029579" watchObservedRunningTime="2026-10-29 22:55:42.049717094 +0000 UTC m=+1339.162362686" Oct 29 22:55:42.415903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:42.415816 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:42.417570 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:42.417541 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:42.417705 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:42.417603 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:42.487863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:42.487258 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:42.487863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:42.487315 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:43.421582 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:43.421545 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:43.422135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:43.421622 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:43.485864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:43.485181 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:43.485864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:43.485238 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:44.415344 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:44.415305 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:44.415570 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:44.415366 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:44.482874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:44.482066 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:44.482874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:44.482124 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:45.415073 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:45.415032 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:45.415249 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:45.415099 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:45.483864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:45.483225 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:45.483864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:45.483287 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:46.416997 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:46.416955 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:46.417220 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:46.417032 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:46.484359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:46.483930 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:46.484359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:46.483994 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:47.415558 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:47.415521 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:47.415738 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:47.415582 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:47.482168 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:47.482133 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 22:55:47.482818 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:47.482787 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 22:55:48.415666 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:48.415630 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 22:55:48.416120 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:48.415715 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 22:55:48.492032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:48.491998 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:48.492621 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:48.492602 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:48.493390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:48.493374 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 22:55:49.420277 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:49.420248 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:49.420815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:49.420794 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:49.421562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:49.421543 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 22:55:54.596287 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:54.596251 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:55:55.466624 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.466591 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch"] Oct 29 22:55:55.472279 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.472257 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.493862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.493804 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-81f92d-openshift-default-dockercfg-x8lwz\"" Oct 29 22:55:55.513155 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.513118 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch"] Oct 29 22:55:55.595752 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.595724 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stp27\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.595914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.595766 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.595914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.595788 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.595914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.595887 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.596045 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.596032 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.596081 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.596067 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.596117 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.596090 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.596230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.596209 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.596267 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.596257 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.697700 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.697670 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.697715 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.697739 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stp27\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.697771 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.697791 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.697813 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698007 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698057 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698164 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698093 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698265 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698402 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698496 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698611 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698515 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.698781 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.698658 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.700229 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.700201 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.700350 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.700322 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.726976 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.726911 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stp27\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.727341 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.727317 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.788057 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.788021 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:55.958301 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.958258 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm"] Oct 29 22:55:55.964061 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.964030 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.026934 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:55.997920 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm"] Oct 29 22:55:56.026934 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.003956 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ffba57-openshift-default-dockercfg-5l7zt\"" Oct 29 22:55:56.026934 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.022116 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms"] Oct 29 22:55:56.033073 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.032151 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.071887 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.071786 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms"] Oct 29 22:55:56.087829 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.087795 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-a2e7c8-openshift-default-dockercfg-lq5n9\"" Oct 29 22:55:56.118890 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.118857 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.118933 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.118997 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119126 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119072 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119158 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119238 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119221 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119254 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119309 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119289 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9cjp\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119422 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119407 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119470 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119456 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119564 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119615 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119596 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119648 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119631 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119765 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119744 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.119799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119780 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119896 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119877 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlsph\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119936 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119909 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.119969 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.119953 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.154275 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.154249 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch"] Oct 29 22:55:56.155600 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:56.155554 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3423324_ad21_4998_a379_6baf63bd8f00.slice/crio-0bca72ae935f3287902885ce6598e426bd457a2d4c7a4de3ff149356b5cba91f WatchSource:0}: Error finding container 0bca72ae935f3287902885ce6598e426bd457a2d4c7a4de3ff149356b5cba91f: Status 404 returned error can't find the container with id 0bca72ae935f3287902885ce6598e426bd457a2d4c7a4de3ff149356b5cba91f Oct 29 22:55:56.166717 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.166670 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:56.166856 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.166762 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:56.166856 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.166810 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:56.221811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.221718 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.221811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.221769 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.221811 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.221809 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlsph\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222075 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.221862 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222075 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.221891 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222075 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.221953 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222075 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222038 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222075 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222062 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222091 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222179 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222261 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222286 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222324 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9cjp\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222340 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222686 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222442 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222686 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222510 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222686 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222600 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222686 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222638 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.222922 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222690 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.222981 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.222955 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.223166 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.223132 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.223518 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.223491 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.224203 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.224182 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.224357 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.224341 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.224676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.224656 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.227545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.226452 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.227545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.226890 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.227545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.227474 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.228264 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.228212 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.229402 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.229368 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.229676 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.229653 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.231996 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.231976 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.254287 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.254239 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.254702 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.254664 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlsph\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.255491 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.255464 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9cjp\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.255640 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.255622 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.293885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.293596 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:56.375950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.374098 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:56.760137 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:55:56.760087 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca490569_fa3a_4182_b39b_810264cbd3e3.slice/crio-16794d13128efac3dbac1ab575b902a0a9384c8437797f75655278d5f52bfcc4 WatchSource:0}: Error finding container 16794d13128efac3dbac1ab575b902a0a9384c8437797f75655278d5f52bfcc4: Status 404 returned error can't find the container with id 16794d13128efac3dbac1ab575b902a0a9384c8437797f75655278d5f52bfcc4 Oct 29 22:55:56.765069 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.763865 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:56.765069 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.763960 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:56.765069 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.764004 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:56.924495 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:56.924417 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm"] Oct 29 22:55:57.438885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.438372 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms"] Oct 29 22:55:57.439775 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.439736 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" event={"ID":"f3423324-ad21-4998-a379-6baf63bd8f00","Type":"ContainerStarted","Data":"7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965"} Oct 29 22:55:57.439918 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.439789 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" event={"ID":"f3423324-ad21-4998-a379-6baf63bd8f00","Type":"ContainerStarted","Data":"0bca72ae935f3287902885ce6598e426bd457a2d4c7a4de3ff149356b5cba91f"} Oct 29 22:55:57.442024 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.441407 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:57.442024 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.441521 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:57.442024 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.441586 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:55:57.575864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.571094 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podStartSLOduration=2.5710579879999997 podStartE2EDuration="2.571057988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:57.565652267 +0000 UTC m=+1354.678297855" watchObservedRunningTime="2026-10-29 22:55:57.571057988 +0000 UTC m=+1354.683703578" Oct 29 22:55:57.617201 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.610079 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" event={"ID":"ca490569-fa3a-4182-b39b-810264cbd3e3","Type":"ContainerStarted","Data":"16794d13128efac3dbac1ab575b902a0a9384c8437797f75655278d5f52bfcc4"} Oct 29 22:55:57.789010 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.788924 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:55:57.789873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.789812 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:55:57.815412 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:57.812130 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:55:58.622439 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.622409 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" event={"ID":"ca490569-fa3a-4182-b39b-810264cbd3e3","Type":"ContainerStarted","Data":"7a00f6411f027715eaf9b3aabf7e8c472fb318b1ba8785acc12c6ae64739f34d"} Oct 29 22:55:58.624236 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.624210 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" event={"ID":"4e9f7211-013a-4d0d-8ffd-71a577250517","Type":"ContainerStarted","Data":"b8fac82d97f566103ca88767fd49eea451e255eb1c1a6b1d6ccdc59517cad63b"} Oct 29 22:55:58.624399 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.624259 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" event={"ID":"4e9f7211-013a-4d0d-8ffd-71a577250517","Type":"ContainerStarted","Data":"22aeffc2d847d18846d3bf6028765306cbc1c5303d5404cc8fb3219da63918f4"} Oct 29 22:55:58.698337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.698285 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podStartSLOduration=3.698265883 podStartE2EDuration="3.698265883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:58.689903012 +0000 UTC m=+1355.802548610" watchObservedRunningTime="2026-10-29 22:55:58.698265883 +0000 UTC m=+1355.810911472" Oct 29 22:55:58.786013 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.785866 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podStartSLOduration=3.785824808 podStartE2EDuration="3.785824808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:55:58.763984771 +0000 UTC m=+1355.876630364" watchObservedRunningTime="2026-10-29 22:55:58.785824808 +0000 UTC m=+1355.898470397" Oct 29 22:55:58.802859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.794215 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:55:58.802859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:58.794278 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:55:59.302864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.301162 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:55:59.302864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.301318 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:55:59.302864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.301363 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:55:59.380861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.380387 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:55:59.380861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.380531 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:55:59.380861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.380573 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:55:59.817863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.810229 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:55:59.817863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:55:59.810292 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:56:00.112917 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.106117 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps"] Oct 29 22:56:00.297866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.294376 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:56:00.297866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.294428 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:56:00.356862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.349078 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps"] Oct 29 22:56:00.395913 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.365109 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.433859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.433146 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-a2e7c8\"/\"openshift-service-ca.crt\"" Oct 29 22:56:00.448715 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.436785 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.516864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.512877 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-a2e7c8\"/\"kube-root-ca.crt\"" Oct 29 22:56:00.563875 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.560296 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a2e7c8\"/\"distinct-2-w2-a2e7c8-kserve-self-signed-certs\"" Oct 29 22:56:00.563875 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.560534 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:56:00.563875 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.560577 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:56:00.582240 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.577807 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a2e7c8\"/\"default-dockercfg-2mj5c\"" Oct 29 22:56:00.730077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.712993 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.730077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.713069 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppz4n\" (UniqueName: \"kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.730077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.713227 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.730077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.713286 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.730077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.713442 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.793352 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.793317 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:56:00.793642 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.793593 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:56:00.818186 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.818143 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.818643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.818273 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.818643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.818313 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.818643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.818364 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppz4n\" (UniqueName: \"kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.818643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.818561 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.818643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.818607 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.833965 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.819489 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.841243 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.840650 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.841701 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.841644 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.867864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.864856 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.881863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.877411 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:00.903862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:00.900524 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppz4n\" (UniqueName: \"kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n\") pod \"distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:01.000390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.000287 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:01.295791 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.295283 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:56:01.295791 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.295343 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:56:01.399988 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.384313 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:56:01.399988 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.384374 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:56:01.840078 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.838403 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:56:01.840078 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:01.838459 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:56:02.165344 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:56:02.165311 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod947f7f47_a7ce_419e_8b65_3f8c74cb608f.slice/crio-eac594cd1acc9d4fabe34f9350d15a85cc653788c16a493a0deb5ce27ef9a189 WatchSource:0}: Error finding container eac594cd1acc9d4fabe34f9350d15a85cc653788c16a493a0deb5ce27ef9a189: Status 404 returned error can't find the container with id eac594cd1acc9d4fabe34f9350d15a85cc653788c16a493a0deb5ce27ef9a189 Oct 29 22:56:02.186354 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.186325 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps"] Oct 29 22:56:02.302231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.302197 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:56:02.302516 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.302478 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:56:02.388304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.387869 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:56:02.388304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.387928 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:56:02.801013 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.800977 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:56:02.801290 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.801251 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:56:02.882040 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:02.882001 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" event={"ID":"947f7f47-a7ce-419e-8b65-3f8c74cb608f","Type":"ContainerStarted","Data":"eac594cd1acc9d4fabe34f9350d15a85cc653788c16a493a0deb5ce27ef9a189"} Oct 29 22:56:03.298229 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.298191 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:56:03.298502 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.298474 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:56:03.375330 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.375296 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:56:03.375627 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.375578 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:56:03.791863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.791219 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 22:56:03.791863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.791280 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 22:56:03.932825 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:03.932782 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" event={"ID":"947f7f47-a7ce-419e-8b65-3f8c74cb608f","Type":"ContainerStarted","Data":"a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3"} Oct 29 22:56:04.024866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:04.006883 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podStartSLOduration=4.006862448 podStartE2EDuration="4.006862448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:56:04.002231638 +0000 UTC m=+1361.114877228" watchObservedRunningTime="2026-10-29 22:56:04.006862448 +0000 UTC m=+1361.119508057" Oct 29 22:56:04.025145 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:04.025099 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:04.306273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:04.306232 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:56:04.306600 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:04.306570 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:56:04.378276 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:04.378239 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:56:04.378551 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:04.378522 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:56:05.139274 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.139243 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:56:05.140563 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.140488 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:56:05.142337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.142291 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 22:56:05.294682 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.294640 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 22:56:05.294865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.294705 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 22:56:05.375022 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.374985 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body= Oct 29 22:56:05.375211 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:05.375051 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" Oct 29 22:56:05.811865 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:56:05.811016 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:56:06.298612 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:06.298583 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:56:06.299102 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:06.299086 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:56:06.299908 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:06.299889 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 22:56:06.379803 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:06.379775 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:56:06.380127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:06.380111 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:56:06.380763 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:06.380744 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 22:56:11.001314 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:11.001282 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:11.017315 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:11.017270 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:12.007771 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:12.007740 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:56:26.447684 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.447652 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq"] Oct 29 22:56:26.453891 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.453865 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.485114 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.485068 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.485459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.485434 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.485699 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.485680 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.486117 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.486078 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.486320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.486302 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.487513 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.487492 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.489976 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.489952 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrrcj\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.490167 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.490149 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.490285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.490269 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.490989 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.490613 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq"] Oct 29 22:56:26.592780 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.592750 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrcj\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.592805 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.592895 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593033 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593085 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593135 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593170 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593212 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593241 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593480 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593421 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593604 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593577 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593645 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.593937 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.593796 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.594320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.594295 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.595557 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.595526 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.595893 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.595875 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.614524 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.614500 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrcj\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.614941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.614924 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token\") pod \"e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:26.787185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:26.787151 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:27.121150 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:27.121074 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq"] Oct 29 22:56:27.127872 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:56:27.127805 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbf9d9f_ffa9_4969_a67c_29d728b914ec.slice/crio-15bd0a85a212dc4dea8078eb2fcb5af3b816b8464cebe0a3477122616f679537 WatchSource:0}: Error finding container 15bd0a85a212dc4dea8078eb2fcb5af3b816b8464cebe0a3477122616f679537: Status 404 returned error can't find the container with id 15bd0a85a212dc4dea8078eb2fcb5af3b816b8464cebe0a3477122616f679537 Oct 29 22:56:27.130013 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:27.129974 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:56:27.130134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:27.130073 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:56:27.130134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:27.130122 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:56:27.920404 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:27.920369 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:56:28.140747 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:28.140708 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" event={"ID":"bdbf9d9f-ffa9-4969-a67c-29d728b914ec","Type":"ContainerStarted","Data":"0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2"} Oct 29 22:56:28.140972 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:28.140766 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" event={"ID":"bdbf9d9f-ffa9-4969-a67c-29d728b914ec","Type":"ContainerStarted","Data":"15bd0a85a212dc4dea8078eb2fcb5af3b816b8464cebe0a3477122616f679537"} Oct 29 22:56:28.225062 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:28.225012 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podStartSLOduration=2.224998958 podStartE2EDuration="2.224998958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:56:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:56:28.209850146 +0000 UTC m=+1385.322495726" watchObservedRunningTime="2026-10-29 22:56:28.224998958 +0000 UTC m=+1385.337644546" Oct 29 22:56:28.795238 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:28.795208 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:28.797174 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:28.797146 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 22:56:28.797408 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:28.797379 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 22:56:29.788295 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:29.788261 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 22:56:29.788690 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:29.788327 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 22:56:30.789867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:30.787942 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 22:56:30.789867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:30.788012 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 22:56:31.788436 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:31.788398 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 22:56:31.788630 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:31.788475 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 22:56:32.788414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:32.788374 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 22:56:32.788831 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:32.788439 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 22:56:33.788356 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:33.788275 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 22:56:33.788356 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:33.788332 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 22:56:34.793468 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:34.793433 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:34.793941 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:34.793831 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:34.794577 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:34.794555 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:56:35.799034 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:56:35.799002 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:56:41.252562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.252531 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs"] Oct 29 22:56:41.257337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.257314 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.303414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.303380 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs"] Oct 29 22:56:41.345599 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.345498 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.345778 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.345621 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7kp7\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.345778 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.345686 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.345930 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.345828 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.345930 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.345891 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.346041 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.345977 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.346041 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.346026 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.346162 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.346144 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.346209 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.346196 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447543 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447509 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447543 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447543 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447720 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447567 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447720 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447591 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447720 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447663 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447720 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447776 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447802 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7kp7\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.447905 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447872 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.448118 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.447975 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.448273 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.448246 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.448397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.448282 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.448397 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.448329 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.448476 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.448463 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.450286 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.450229 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.450461 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.450441 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.469661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.469631 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.470313 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.470291 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7kp7\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7\") pod \"e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.574429 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.574343 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:41.910983 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.910957 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs"] Oct 29 22:56:41.912071 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:56:41.912048 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec73450_2334_40a8_a461_20fa18c57542.slice/crio-f15cf1cd048a7d41a614f7cb42b8b9bb505115957380b1274bc63c3a879686db WatchSource:0}: Error finding container f15cf1cd048a7d41a614f7cb42b8b9bb505115957380b1274bc63c3a879686db: Status 404 returned error can't find the container with id f15cf1cd048a7d41a614f7cb42b8b9bb505115957380b1274bc63c3a879686db Oct 29 22:56:41.914644 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.914605 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:56:41.914736 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.914677 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:56:41.914736 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:41.914710 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:56:42.276249 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:42.276210 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" event={"ID":"dec73450-2334-40a8-a461-20fa18c57542","Type":"ContainerStarted","Data":"ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c"} Oct 29 22:56:42.276770 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:42.276275 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" event={"ID":"dec73450-2334-40a8-a461-20fa18c57542","Type":"ContainerStarted","Data":"f15cf1cd048a7d41a614f7cb42b8b9bb505115957380b1274bc63c3a879686db"} Oct 29 22:56:42.323789 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:42.323742 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podStartSLOduration=1.323728675 podStartE2EDuration="1.323728675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:56:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:56:42.322183204 +0000 UTC m=+1399.434828803" watchObservedRunningTime="2026-10-29 22:56:42.323728675 +0000 UTC m=+1399.436374263" Oct 29 22:56:42.581918 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:42.581063 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:42.583160 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:42.583126 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:42.583345 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:42.583186 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:43.582865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:43.582553 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:43.582865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:43.582608 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:44.578245 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:44.578207 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:44.578520 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:44.578490 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:45.574901 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:45.574864 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:45.575490 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:45.575458 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:46.575866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:46.575208 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:46.575866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:46.575269 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:47.575222 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:47.575183 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:47.575519 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:47.575491 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:48.575473 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:48.575431 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 22:56:48.576032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:48.575498 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 22:56:50.629863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:50.628956 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:56:50.629863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:50.629035 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:56:51.673988 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:51.673916 5030 patch_prober.go:29] interesting pod/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:56:51.674659 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:51.674597 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:56:52.347994 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:52.347952 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:52.411865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:52.403730 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:52.417481 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:52.417453 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 22:56:52.464295 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:52.463979 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:56:52.464295 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:56:52.464049 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:03.755821 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:03.755789 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:57:05.972600 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:57:05.972536 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:57:11.235776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.235738 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9"] Oct 29 22:57:11.240213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.240194 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.275435 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.275403 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9"] Oct 29 22:57:11.354690 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.354648 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.354909 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.354715 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.354909 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.354743 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm5j9\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.355044 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.354914 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.355145 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.355124 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.355198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.355187 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.355312 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.355285 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.355370 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.355354 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.355420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.355388 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457322 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457281 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457367 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457423 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457506 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457492 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457527 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457683 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457614 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457790 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457761 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.457998 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457969 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.458131 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.458021 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.458131 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.457650 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.458131 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.458114 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm5j9\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.458299 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.458169 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.459368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.458572 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.459368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.458608 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.460640 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.460621 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.460753 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.460742 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.484691 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.484662 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm5j9\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.490359 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.490288 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.557815 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.557783 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:11.895301 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.895272 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9"] Oct 29 22:57:11.896237 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:57:11.896203 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca5ff35_abae_4a7a_8077_fdafd607699a.slice/crio-ed041b71072135e412c8d710f46b014043cdd568dba38d9c9d64b8a0cb3ecf58 WatchSource:0}: Error finding container ed041b71072135e412c8d710f46b014043cdd568dba38d9c9d64b8a0cb3ecf58: Status 404 returned error can't find the container with id ed041b71072135e412c8d710f46b014043cdd568dba38d9c9d64b8a0cb3ecf58 Oct 29 22:57:11.899258 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.899220 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:11.899339 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.899287 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:11.899339 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:11.899317 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:12.686871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:12.685829 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" event={"ID":"fca5ff35-abae-4a7a-8077-fdafd607699a","Type":"ContainerStarted","Data":"2ae8ab1eb31dcccb4d7b4dac910e5cb8a7eebf1a112ef4f05390ae07a9765097"} Oct 29 22:57:12.686871 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:12.685985 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" event={"ID":"fca5ff35-abae-4a7a-8077-fdafd607699a","Type":"ContainerStarted","Data":"ed041b71072135e412c8d710f46b014043cdd568dba38d9c9d64b8a0cb3ecf58"} Oct 29 22:57:12.771864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:12.770919 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podStartSLOduration=1.7708980909999998 podStartE2EDuration="1.770898091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:57:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:57:12.763529024 +0000 UTC m=+1429.876174612" watchObservedRunningTime="2026-10-29 22:57:12.770898091 +0000 UTC m=+1429.883543697" Oct 29 22:57:13.568874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:13.566128 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:13.568874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:13.567178 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 22:57:13.568874 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:13.567239 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 22:57:14.572279 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:14.562342 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 22:57:14.572279 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:14.562403 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 22:57:15.561008 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:15.560973 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 22:57:15.561299 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:15.561256 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 22:57:16.558248 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:16.558196 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 22:57:16.558861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:16.558809 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 22:57:17.558674 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:17.558203 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 22:57:17.558674 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:17.558267 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 22:57:18.558460 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:18.558422 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 22:57:18.558662 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:18.558492 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 22:57:19.562698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:19.562669 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:19.563193 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:19.563114 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:19.563821 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:19.563805 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 22:57:25.779232 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.779196 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq"] Oct 29 22:57:25.786325 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.786301 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.812776 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.807507 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq"] Oct 29 22:57:25.912954 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.912926 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913124 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.912975 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913124 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913102 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913189 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913238 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913212 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913351 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913337 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913389 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913375 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913464 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913447 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:25.913576 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:25.913559 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vzdr\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.014919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.014891 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.014919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.014926 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.014991 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015035 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015073 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015257 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015162 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzdr\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015257 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015209 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015358 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015261 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015358 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015328 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015474 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015331 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015474 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015375 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015583 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015738 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.015989 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.015795 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.017779 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.017759 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.017947 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.017932 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.037708 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.037649 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.037821 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.037805 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vzdr\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.106995 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.106971 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:26.332487 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.332414 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h"] Oct 29 22:57:26.338690 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.338665 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.367464 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.367295 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h"] Oct 29 22:57:26.401474 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.401439 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq"] Oct 29 22:57:26.407219 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.407196 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.426990 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427059 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427134 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427175 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427213 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427239 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcx8j\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427294 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427411 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.427797 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.427586 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.440573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.440542 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq"] Oct 29 22:57:26.455956 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.455925 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq"] Oct 29 22:57:26.467554 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.467517 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:26.467677 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.467603 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:26.467677 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.467638 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:26.529007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.528972 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529110 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529012 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529154 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529115 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529264 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529249 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529343 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529322 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529432 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529411 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529472 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529557 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529499 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdz4\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529557 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529500 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529557 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529523 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529557 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529543 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529567 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529582 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcx8j\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529612 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529637 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.529766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529699 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.529766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529733 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.530087 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529778 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.530087 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529610 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.530087 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529799 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.530087 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.529912 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.530228 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.530090 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.530377 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.530360 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.532313 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.532293 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.532594 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.532575 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.552575 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.552550 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcx8j\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.553450 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.553419 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.631375 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631238 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631375 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631368 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631597 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631471 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdz4\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631597 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631507 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631597 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631557 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631742 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631649 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631742 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631735 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631860 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631809 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.631922 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631881 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.632009 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.631988 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.632239 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.632211 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.632313 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.632279 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.632370 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.632355 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.632667 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.632649 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.634407 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.634387 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.634524 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.634463 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.654465 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.654441 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.655526 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.655502 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdz4\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4\") pod \"e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.663452 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.663434 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:26.726602 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.726364 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:26.814923 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.814313 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" event={"ID":"191a1836-d335-4756-aa1e-fbc676effe2b","Type":"ContainerStarted","Data":"d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16"} Oct 29 22:57:26.814923 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.814383 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" event={"ID":"191a1836-d335-4756-aa1e-fbc676effe2b","Type":"ContainerStarted","Data":"fb0d07c47ba2364bea0b5639f748e324aee68d6d3c42dd548954af0e03da93b1"} Oct 29 22:57:26.863204 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:26.862696 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podStartSLOduration=1.862676069 podStartE2EDuration="1.862676069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:57:26.861291896 +0000 UTC m=+1443.973937496" watchObservedRunningTime="2026-10-29 22:57:26.862676069 +0000 UTC m=+1443.975321661" Oct 29 22:57:27.018920 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.014931 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h"] Oct 29 22:57:27.020807 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:57:27.020775 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b8b321_58ef_4dea_a685_7f17127a228a.slice/crio-4c183acf25f1916ae0cd99181d572e59f09e39ce0765d5e051ceb2f7b80c8165 WatchSource:0}: Error finding container 4c183acf25f1916ae0cd99181d572e59f09e39ce0765d5e051ceb2f7b80c8165: Status 404 returned error can't find the container with id 4c183acf25f1916ae0cd99181d572e59f09e39ce0765d5e051ceb2f7b80c8165 Oct 29 22:57:27.023810 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.023776 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:27.023925 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.023893 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:27.023998 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.023946 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:27.085512 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:57:27.085466 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7133acf0_63a8_4436_bda0_a9961775d342.slice/crio-484c2e95a6d069c999cd6c146951889a2f7fe15de2f983de5cd631cfa7a54b37 WatchSource:0}: Error finding container 484c2e95a6d069c999cd6c146951889a2f7fe15de2f983de5cd631cfa7a54b37: Status 404 returned error can't find the container with id 484c2e95a6d069c999cd6c146951889a2f7fe15de2f983de5cd631cfa7a54b37 Oct 29 22:57:27.087305 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.087119 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq"] Oct 29 22:57:27.089758 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.089718 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:27.089907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.089826 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:27.089907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.089897 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:27.107291 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.107233 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:27.109445 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.109417 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:27.109554 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.109453 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:27.824893 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.824825 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" event={"ID":"7133acf0-63a8-4436-bda0-a9961775d342","Type":"ContainerStarted","Data":"dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3"} Oct 29 22:57:27.825606 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.824912 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" event={"ID":"7133acf0-63a8-4436-bda0-a9961775d342","Type":"ContainerStarted","Data":"484c2e95a6d069c999cd6c146951889a2f7fe15de2f983de5cd631cfa7a54b37"} Oct 29 22:57:27.826663 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.826632 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" event={"ID":"a8b8b321-58ef-4dea-a685-7f17127a228a","Type":"ContainerStarted","Data":"bdb6d1240d9f7cee759899fdd2888971052f3e2d06d2dbd3ce97a27da4b4f06a"} Oct 29 22:57:27.826772 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.826678 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" event={"ID":"a8b8b321-58ef-4dea-a685-7f17127a228a","Type":"ContainerStarted","Data":"4c183acf25f1916ae0cd99181d572e59f09e39ce0765d5e051ceb2f7b80c8165"} Oct 29 22:57:27.930659 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:27.930595 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podStartSLOduration=1.930572567 podStartE2EDuration="1.930572567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:57:27.882045697 +0000 UTC m=+1444.994691302" watchObservedRunningTime="2026-10-29 22:57:27.930572567 +0000 UTC m=+1445.043218156" Oct 29 22:57:28.127861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.127222 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:28.127861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.127283 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:28.671912 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.669435 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:28.671912 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.669600 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:28.671912 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.669649 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:28.736420 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.735994 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:28.736616 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.736502 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:28.736616 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:28.736558 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:29.125865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:29.111333 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:29.125865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:29.111407 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:29.672865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:29.668750 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:29.672865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:29.668814 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:29.758862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:29.751985 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:29.758862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:29.752065 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:30.112171 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.112136 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:30.112450 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.112419 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:30.188007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.187946 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:30.667873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.667232 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:30.667873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.667295 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:30.739908 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.739854 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:30.740230 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.740185 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:30.918039 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.917939 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded" Oct 29 22:57:30.999823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:30.999778 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:31.015634 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.015595 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:31.249928 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.213474 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded" Oct 29 22:57:31.249928 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.214124 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:57:31.250748 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.250718 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:31.250968 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.250938 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:31.348291 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.348224 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podStartSLOduration=5.348200423 podStartE2EDuration="5.348200423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:57:27.936763952 +0000 UTC m=+1445.049409543" watchObservedRunningTime="2026-10-29 22:57:31.348200423 +0000 UTC m=+1448.460846013" Oct 29 22:57:31.749077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.749044 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:31.749443 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.749416 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:31.749950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.749353 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:31.750133 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.750113 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:31.942992 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.942941 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:31.952967 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:31.945476 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:57:32.010931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.003277 5030 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:32.010931 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.003510 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded" Oct 29 22:57:32.013023 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.013000 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:57:32.031542 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.031503 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:32.043876 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.033133 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:57:32.240940 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.240896 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:32.428337 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.428218 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:32.466039 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.465992 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:32.672566 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.672522 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:32.678973 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.672790 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:32.757050 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.757013 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:32.764589 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.764550 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:32.979977 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:32.970927 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded" Oct 29 22:57:33.081984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.081937 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:33.183867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.082237 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded" Oct 29 22:57:33.194245 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.160965 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:33.209963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.209918 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:33.247059 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.246958 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:33.742614 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.742583 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:33.780486 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.780442 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:33.801020 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.800955 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:33.918998 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.897033 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:33.984234 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:33.984187 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:34.124950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.121075 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:34.124950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.121134 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:34.186339 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.168254 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:34.199866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.194266 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:34.260873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.259126 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded" Oct 29 22:57:34.581863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.579867 5030 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:34.699298 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.699265 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:34.699793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.699753 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:34.786880 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.763132 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:34.786880 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.763186 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:34.993872 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:34.992209 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:35.108007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.107971 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:35.108331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.108282 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:35.168978 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.168931 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:35.261991 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.257932 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:35.262578 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.262239 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded" Oct 29 22:57:35.665923 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.665173 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:35.665923 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.665233 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:35.764863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.754724 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:57:35.785866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.766388 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:35.785866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.766443 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:35.930592 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.930516 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:57:35.968320 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.968262 5030 patch_prober.go:29] interesting pod/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:57:35.968590 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.968562 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:35.974711 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.974684 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:57:35.978208 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:35.978188 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:57:36.130876 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:36.126090 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:36.130876 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:36.126183 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:36.699866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:36.693996 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:36.699866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:36.694074 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:36.800907 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:36.800872 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:36.801231 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:36.801183 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:37.011611 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:57:37.011572 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:57:37.133794 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.133748 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:37.134136 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.134105 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:37.186924 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.178336 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:37.268929 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.267709 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded" Oct 29 22:57:37.669913 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.666211 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:37.669913 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.666272 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:37.727501 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.727472 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:37.727686 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:37.727539 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:38.107442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:38.107407 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 22:57:38.108079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:38.108046 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 22:57:38.664819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:38.664732 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 29 22:57:38.664819 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:38.664790 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 29 22:57:38.728032 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:38.727994 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 22:57:38.728363 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:38.728327 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 22:57:40.162903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.159932 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:57:40.162903 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.159990 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:40.718937 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.717967 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:57:40.718937 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.718043 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:40.769969 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.769930 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:57:40.770241 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.770211 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:40.888508 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.888464 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn"] Oct 29 22:57:40.938703 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.938671 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.957760 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.957667 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn"] Oct 29 22:57:40.973908 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.973813 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.974199 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.974175 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t5wf\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.974475 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.974439 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.974766 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.974737 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.974902 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.974816 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.974976 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.974957 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.975039 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.974999 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.975101 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.975070 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:40.975160 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:40.975105 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.081006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.077469 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.081006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.077523 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.081006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.077604 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.081006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.077642 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.081006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.077747 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.081006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.077817 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t5wf\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.101689 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.101824 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.101919 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.102502 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.103199 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.103449 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.105323 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.122127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.105609 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.136380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.125092 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:41.136380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.125352 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:41.136380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.126653 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:41.136380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.129219 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 22:57:41.136380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.130780 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.136380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.133729 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t5wf\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.139501 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.139477 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.139979 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.139906 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy\") pod \"e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.162122 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.161059 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:41.303327 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.302815 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:41.436383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.435004 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:41.436383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.436313 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:41.438415 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.438375 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 22:57:41.678557 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:57:41.678499 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e01f27_9df6_4157_b1d8_77a71ae0f62d.slice/crio-a2b86bc7e107f1d10fc705f61aa995ad175f5c6dfa1893c10228219a6c23a855 WatchSource:0}: Error finding container a2b86bc7e107f1d10fc705f61aa995ad175f5c6dfa1893c10228219a6c23a855: Status 404 returned error can't find the container with id a2b86bc7e107f1d10fc705f61aa995ad175f5c6dfa1893c10228219a6c23a855 Oct 29 22:57:41.679971 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.679945 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn"] Oct 29 22:57:41.682340 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.682296 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:41.682471 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.682400 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:41.682471 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:41.682455 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:42.136052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.136015 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" event={"ID":"75e01f27-9df6-4157-b1d8-77a71ae0f62d","Type":"ContainerStarted","Data":"0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92"} Oct 29 22:57:42.136245 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.136081 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" event={"ID":"75e01f27-9df6-4157-b1d8-77a71ae0f62d","Type":"ContainerStarted","Data":"a2b86bc7e107f1d10fc705f61aa995ad175f5c6dfa1893c10228219a6c23a855"} Oct 29 22:57:42.137966 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.137938 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 22:57:42.191865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.187931 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podStartSLOduration=2.187911451 podStartE2EDuration="2.187911451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:57:42.186470101 +0000 UTC m=+1459.299115690" watchObservedRunningTime="2026-10-29 22:57:42.187911451 +0000 UTC m=+1459.300557041" Oct 29 22:57:42.303886 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.303828 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:42.305551 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.305522 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:42.305767 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:42.305576 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:43.303553 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:43.303518 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:43.303732 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:43.303581 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:44.303523 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:44.303495 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:44.303956 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:44.303538 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:45.303631 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:45.303601 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:45.304017 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:45.303652 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:46.303680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:46.303639 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:46.304063 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:46.303708 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:47.303395 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:47.303359 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:47.303588 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:47.303422 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:48.303209 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:48.303178 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 29 22:57:48.303629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:48.303227 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 29 22:57:49.308967 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:49.308935 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:49.309529 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:49.309509 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:49.310260 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:49.310241 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 22:57:56.169311 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.169280 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:57:56.400733 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.400652 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd"] Oct 29 22:57:56.408098 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.408063 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.456963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.456888 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd"] Oct 29 22:57:56.581288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581246 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581407 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581466 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581641 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581626 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581777 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581833 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581819 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581833 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581857 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg5hl\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.581994 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.581972 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.582065 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.582049 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683353 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683422 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683454 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg5hl\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683510 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683560 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683592 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683676 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683799 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683735 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683987 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683806 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.683987 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.683822 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.684175 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.684147 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.684175 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.684186 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.684175 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.684250 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.684613 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.684352 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.686079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.686056 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.686310 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.686294 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.716646 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.716623 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg5hl\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.718483 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.718458 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token\") pod \"e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:56.727402 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:56.727383 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:57.083664 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:57.081697 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd"] Oct 29 22:57:57.095779 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:57:57.095705 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4ec2cb_2934_4f04_95e0_dd06b21cfa71.slice/crio-82b3feb5ced308e466b7521004e84fc484676e7d36fbd1ae93683ceefe84fc93 WatchSource:0}: Error finding container 82b3feb5ced308e466b7521004e84fc484676e7d36fbd1ae93683ceefe84fc93: Status 404 returned error can't find the container with id 82b3feb5ced308e466b7521004e84fc484676e7d36fbd1ae93683ceefe84fc93 Oct 29 22:57:57.173746 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:57.158247 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:57.173746 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:57.158339 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:57.173746 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:57.158411 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:57:57.371082 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:57.370984 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" event={"ID":"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71","Type":"ContainerStarted","Data":"82b3feb5ced308e466b7521004e84fc484676e7d36fbd1ae93683ceefe84fc93"} Oct 29 22:57:58.502866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:58.501686 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" event={"ID":"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71","Type":"ContainerStarted","Data":"b23ba6deb4e89040189294d68c7659f88512e35e3f986eea5e4a08be87a5dec6"} Oct 29 22:57:58.657873 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:58.643083 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podStartSLOduration=2.64306131 podStartE2EDuration="2.64306131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:57:58.595392722 +0000 UTC m=+1475.708038361" watchObservedRunningTime="2026-10-29 22:57:58.64306131 +0000 UTC m=+1475.755706899" Oct 29 22:57:58.734867 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:58.734753 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:57:58.735074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:58.734957 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:57:58.735074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:58.735006 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:57:59.080984 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:59.080937 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:59.218531 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:59.194038 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:59.218531 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:59.212935 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:57:59.744515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:59.744479 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:57:59.745189 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:57:59.745157 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:00.260861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:00.250231 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:00.260861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:00.257857 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:00.260861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:00.259712 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:58:00.260861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:00.259794 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:58:00.798979 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:00.738957 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:00.798979 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:00.739012 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:01.315865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.304087 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded" Oct 29 22:58:01.315865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.304491 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded" Oct 29 22:58:01.411914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.371879 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded" Oct 29 22:58:01.411914 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.372258 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded" Oct 29 22:58:01.641919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.570181 5030 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:01.816880 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.802506 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:01.816880 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:01.802565 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:02.026272 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.026221 5030 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:02.305019 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.304922 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:02.426861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.312258 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded" Oct 29 22:58:02.426861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.425168 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:02.426861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.426706 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:58:02.426861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.426780 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:58:02.451378 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.425206 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:02.833332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.734252 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:02.833332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:02.734311 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:03.312170 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:03.312121 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:03.433919 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:03.432563 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:03.488865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:03.486967 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": context deadline exceeded" Oct 29 22:58:03.488865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:03.487535 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded" Oct 29 22:58:03.796866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:03.761287 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:03.796866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:03.761352 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:04.430134 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:04.430103 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 22:58:04.444116 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:04.435114 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 22:58:04.488968 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:04.488937 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 22:58:04.520885 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:04.519853 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 22:58:04.729115 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:04.729082 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:04.729295 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:04.729153 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:05.728028 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:05.727992 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:05.728488 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:05.728055 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:05.830976 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:05.830937 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:58:06.728673 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:06.728642 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:06.729110 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:06.728691 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:07.728383 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:07.728345 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:07.728559 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:07.728408 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:08.728562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:08.728519 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 29 22:58:08.729071 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:08.728587 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 29 22:58:09.218733 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.218701 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9038-1e1933"] Oct 29 22:58:09.224698 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.224678 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:09.241701 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.241678 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-p5pkd\"" Oct 29 22:58:09.270216 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.270185 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9038-1e1933"] Oct 29 22:58:09.366409 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.366364 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgkn6\" (UniqueName: \"kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6\") pod \"test-curl-9038-1e1933\" (UID: \"725836c1-b62d-4701-a109-cbb8db72bc4f\") " pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:09.469259 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.469125 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgkn6\" (UniqueName: \"kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6\") pod \"test-curl-9038-1e1933\" (UID: \"725836c1-b62d-4701-a109-cbb8db72bc4f\") " pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:09.505314 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.505283 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgkn6\" (UniqueName: \"kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6\") pod \"test-curl-9038-1e1933\" (UID: \"725836c1-b62d-4701-a109-cbb8db72bc4f\") " pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:09.539414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.539388 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:09.871112 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.868377 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9038-1e1933"] Oct 29 22:58:09.871112 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:58:09.869647 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod725836c1_b62d_4701_a109_cbb8db72bc4f.slice/crio-ee186e3db8321a70625e6ac19f5b994618cb0ca3b0d56886c0e9d7e13943e948 WatchSource:0}: Error finding container ee186e3db8321a70625e6ac19f5b994618cb0ca3b0d56886c0e9d7e13943e948: Status 404 returned error can't find the container with id ee186e3db8321a70625e6ac19f5b994618cb0ca3b0d56886c0e9d7e13943e948 Oct 29 22:58:09.872053 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.872032 5030 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 22:58:09.954464 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.954424 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:58:09.955050 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.955025 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:58:09.956675 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:09.956651 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 22:58:10.532855 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:10.532805 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9038-1e1933" event={"ID":"725836c1-b62d-4701-a109-cbb8db72bc4f","Type":"ContainerStarted","Data":"ee186e3db8321a70625e6ac19f5b994618cb0ca3b0d56886c0e9d7e13943e948"} Oct 29 22:58:12.573758 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:12.559203 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9038-1e1933" event={"ID":"725836c1-b62d-4701-a109-cbb8db72bc4f","Type":"ContainerStarted","Data":"e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328"} Oct 29 22:58:12.625859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:12.606242 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9038-1e1933" podStartSLOduration=1.52450608 podStartE2EDuration="3.60622365s" totalImagesPullingTime="2.08171757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:58:12.601302875 +0000 UTC m=+1489.713948463" watchObservedRunningTime="2026-10-29 22:58:12.60622365 +0000 UTC m=+1489.718869240" Oct 29 22:58:13.624975 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.624942 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9038-1e1933"] Oct 29 22:58:13.630201 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.630170 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9038-1e1933"] Oct 29 22:58:13.797152 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.797120 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9038-bb78e1"] Oct 29 22:58:13.797912 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.797896 5030 cpu_manager.go:414] "containerMap: removing container" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" containerName="test-curl-9038-1e1933" Oct 29 22:58:13.797971 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.797913 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" containerName="test-curl-9038-1e1933" Oct 29 22:58:13.798079 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.798068 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" containerName="test-curl-9038-1e1933" Oct 29 22:58:13.802005 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.801989 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:13.818474 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.818454 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9038-bb78e1"] Oct 29 22:58:13.955014 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:13.954931 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5cnf\" (UniqueName: \"kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf\") pod \"test-curl-9038-bb78e1\" (UID: \"10fca084-2b17-4078-a668-bf8a73c32acb\") " pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:14.056643 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.056610 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5cnf\" (UniqueName: \"kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf\") pod \"test-curl-9038-bb78e1\" (UID: \"10fca084-2b17-4078-a668-bf8a73c32acb\") " pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:14.078329 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.078299 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5cnf\" (UniqueName: \"kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf\") pod \"test-curl-9038-bb78e1\" (UID: \"10fca084-2b17-4078-a668-bf8a73c32acb\") " pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:14.115546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.115516 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:14.431866 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.431817 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9038-bb78e1"] Oct 29 22:58:14.437641 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:58:14.437604 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10fca084_2b17_4078_a668_bf8a73c32acb.slice/crio-8a837d4c01e4125a36661a6bad5ce56ee768458dd501de40cc8bd8d0a4c847c8 WatchSource:0}: Error finding container 8a837d4c01e4125a36661a6bad5ce56ee768458dd501de40cc8bd8d0a4c847c8: Status 404 returned error can't find the container with id 8a837d4c01e4125a36661a6bad5ce56ee768458dd501de40cc8bd8d0a4c847c8 Oct 29 22:58:14.576433 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.576401 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9038-bb78e1" event={"ID":"10fca084-2b17-4078-a668-bf8a73c32acb","Type":"ContainerStarted","Data":"aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc"} Oct 29 22:58:14.576556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.576444 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9038-bb78e1" event={"ID":"10fca084-2b17-4078-a668-bf8a73c32acb","Type":"ContainerStarted","Data":"8a837d4c01e4125a36661a6bad5ce56ee768458dd501de40cc8bd8d0a4c847c8"} Oct 29 22:58:14.576556 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.576500 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9038-1e1933" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" containerName="test-curl-9038-1e1933" containerID="cri-o://e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328" gracePeriod=2 Oct 29 22:58:14.626477 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:14.626431 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9038-bb78e1" podStartSLOduration=1.62641864 podStartE2EDuration="1.62641864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:58:14.623499942 +0000 UTC m=+1491.736145530" watchObservedRunningTime="2026-10-29 22:58:14.62641864 +0000 UTC m=+1491.739064228" Oct 29 22:58:14.640205 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:14.640180 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9038-1e1933\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:14.656115 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:14.656089 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9038-1e1933\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:15.593332 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:15.593295 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9038-bb78e1"] Oct 29 22:58:15.600568 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:15.600535 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9038-bb78e1"] Oct 29 22:58:16.602147 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:16.600565 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9038-bb78e1" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" containerName="test-curl-9038-bb78e1" containerID="cri-o://aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc" gracePeriod=2 Oct 29 22:58:16.612608 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:16.612569 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9038-bb78e1\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:16.833368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:16.833309 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:16.848698 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:16.848662 5030 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9038-bb78e1\" is forbidden: User \"system:node:ip-10-0-136-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-228.ec2.internal' and this object" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:16.905280 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:16.905238 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6\" (UniqueName: \"kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6\") pod \"725836c1-b62d-4701-a109-cbb8db72bc4f\" (UID: \"725836c1-b62d-4701-a109-cbb8db72bc4f\") " Oct 29 22:58:16.908637 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:16.907711 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6" pod "725836c1-b62d-4701-a109-cbb8db72bc4f" (UID: "725836c1-b62d-4701-a109-cbb8db72bc4f"). InnerVolumeSpecName "kube-api-access-dgkn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:58:17.008562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.008074 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgkn6\" (UniqueName: \"kubernetes.io/projected/725836c1-b62d-4701-a109-cbb8db72bc4f-kube-api-access-dgkn6\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:58:17.119810 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.119730 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="725836c1-b62d-4701-a109-cbb8db72bc4f" path="/var/lib/kubelet/pods/725836c1-b62d-4701-a109-cbb8db72bc4f/volumes" Oct 29 22:58:17.606987 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.606950 5030 generic.go:356] "Generic (PLEG): container finished" podID="725836c1-b62d-4701-a109-cbb8db72bc4f" containerID="e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328" exitCode=137 Oct 29 22:58:17.607463 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.607020 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-1e1933" Oct 29 22:58:17.607463 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.607054 5030 scope.go:122] "RemoveContainer" containerID="e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328" Oct 29 22:58:17.619000 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.618976 5030 scope.go:122] "RemoveContainer" containerID="e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328" Oct 29 22:58:17.619337 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:17.619305 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328\": container with ID starting with e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328 not found: ID does not exist" containerID="e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328" Oct 29 22:58:17.619454 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:17.619347 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328"} err="failed to get container status \"e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328\": rpc error: code = NotFound desc = could not find container \"e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328\": container with ID starting with e882a3313ebddb1673d3b6764ebafaa86c9a92f3071657ea9ee60f0f36e6d328 not found: ID does not exist" Oct 29 22:58:18.844117 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:18.844093 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:18.952019 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:18.951911 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf\" (UniqueName: \"kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf\") pod \"10fca084-2b17-4078-a668-bf8a73c32acb\" (UID: \"10fca084-2b17-4078-a668-bf8a73c32acb\") " Oct 29 22:58:18.969057 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:18.969016 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf" pod "10fca084-2b17-4078-a668-bf8a73c32acb" (UID: "10fca084-2b17-4078-a668-bf8a73c32acb"). InnerVolumeSpecName "kube-api-access-d5cnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:58:19.054545 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.054515 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5cnf\" (UniqueName: \"kubernetes.io/projected/10fca084-2b17-4078-a668-bf8a73c32acb-kube-api-access-d5cnf\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:58:19.056856 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.056819 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" path="/var/lib/kubelet/pods/10fca084-2b17-4078-a668-bf8a73c32acb/volumes" Oct 29 22:58:19.628289 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.628259 5030 generic.go:356] "Generic (PLEG): container finished" podID="10fca084-2b17-4078-a668-bf8a73c32acb" containerID="aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc" exitCode=137 Oct 29 22:58:19.628478 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.628298 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9038-bb78e1" Oct 29 22:58:19.628478 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.628354 5030 scope.go:122] "RemoveContainer" containerID="aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc" Oct 29 22:58:19.638716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.638697 5030 scope.go:122] "RemoveContainer" containerID="aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc" Oct 29 22:58:19.638991 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:19.638971 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc\": container with ID starting with aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc not found: ID does not exist" containerID="aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc" Oct 29 22:58:19.639091 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:19.639006 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc"} err="failed to get container status \"aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc\": rpc error: code = NotFound desc = could not find container \"aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc\": container with ID starting with aef1e7635e422e4e230283dc102b244f957735883fa38868a7e8ca59c39469fc not found: ID does not exist" Oct 29 22:58:21.475226 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.475198 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624"] Oct 29 22:58:21.475981 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.475964 5030 cpu_manager.go:414] "containerMap: removing container" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" containerName="test-curl-9038-bb78e1" Oct 29 22:58:21.475981 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.475982 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" containerName="test-curl-9038-bb78e1" Oct 29 22:58:21.476113 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.476103 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="10fca084-2b17-4078-a668-bf8a73c32acb" containerName="test-curl-9038-bb78e1" Oct 29 22:58:21.481567 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.481547 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.500346 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.500321 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624"] Oct 29 22:58:21.509546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.509520 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2bd3f07e-dockercfg-s9gzd\"" Oct 29 22:58:21.509666 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.509624 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2bd3f07e\"" Oct 29 22:58:21.613650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.602891 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.613650 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.603134 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmj6b\" (UniqueName: \"kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b\") pod \"payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.631909 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.629047 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz"] Oct 29 22:58:21.704750 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.704716 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.710292 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.705702 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.710292 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.705910 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz"] Oct 29 22:58:21.710292 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.706013 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:21.723792 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.713983 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmj6b\" (UniqueName: \"kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b\") pod \"payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.749403 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.749288 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmj6b\" (UniqueName: \"kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b\") pod \"payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.799589 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.799556 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:21.815390 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.815364 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzzbz\" (UniqueName: \"kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz\") pod \"payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:21.815562 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.815535 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume\") pod \"payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:21.916787 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.916752 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume\") pod \"payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:21.917029 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.917010 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzzbz\" (UniqueName: \"kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz\") pod \"payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:21.917790 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.917764 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume\") pod \"payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:21.940540 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:21.940511 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzzbz\" (UniqueName: \"kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz\") pod \"payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:22.041965 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:22.041884 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:22.148127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:22.148100 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624"] Oct 29 22:58:22.358892 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:22.358866 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz"] Oct 29 22:58:22.360778 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:58:22.360755 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa5f038_4ebd_45c4_b2f5_7074221c8d9c.slice/crio-729ce4ade7e9878f4e5f6838cac5da396e203517c0103f4e29a618a93b0d21a3 WatchSource:0}: Error finding container 729ce4ade7e9878f4e5f6838cac5da396e203517c0103f4e29a618a93b0d21a3: Status 404 returned error can't find the container with id 729ce4ade7e9878f4e5f6838cac5da396e203517c0103f4e29a618a93b0d21a3 Oct 29 22:58:22.706981 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:22.706734 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" event={"ID":"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c","Type":"ContainerStarted","Data":"729ce4ade7e9878f4e5f6838cac5da396e203517c0103f4e29a618a93b0d21a3"} Oct 29 22:58:22.730238 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:22.730196 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" event={"ID":"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee","Type":"ContainerStarted","Data":"5f9a299a07509f4fdd437239369ee4c3555e29f3a00e713efd7cf98b9591ae0e"} Oct 29 22:58:24.757865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:24.757005 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:58:24.782861 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:24.779122 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 22:58:26.029863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:26.015279 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:58:26.029863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:26.015344 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:26.325031 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:26.324924 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:30.056661 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.056628 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk"] Oct 29 22:58:30.061705 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.061681 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.086740 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.086716 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk"] Oct 29 22:58:30.095849 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.095809 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-01e33b3d\"" Oct 29 22:58:30.095985 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.095956 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-01e33b3d-dockercfg-266tp\"" Oct 29 22:58:30.171660 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.171556 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.172380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.172349 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx4rc\" (UniqueName: \"kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc\") pod \"payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.291555 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.289312 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx4rc\" (UniqueName: \"kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc\") pod \"payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.291555 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.289429 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.293910 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.293871 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.295192 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.294812 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" event={"ID":"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c","Type":"ContainerStarted","Data":"af54dfea1cc708b227d09ff5ed1c00d1e300f960f0dca24114e039a8ab999448"} Oct 29 22:58:30.295192 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.294959 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:30.297898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.297858 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" event={"ID":"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee","Type":"ContainerStarted","Data":"738bef8b0b514a38cbb65e5a6d533b23ef544dc796d277b51d718ae0de0b16d5"} Oct 29 22:58:30.298424 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.298402 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:30.370319 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.370248 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx4rc\" (UniqueName: \"kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc\") pod \"payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.376202 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.376168 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:30.421913 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.416058 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" podStartSLOduration=2.592941724 podStartE2EDuration="9.416038713s" totalImagesPullingTime="6.823096989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:58:30.412758401 +0000 UTC m=+1507.525403988" watchObservedRunningTime="2026-10-29 22:58:30.416038713 +0000 UTC m=+1507.528684302" Oct 29 22:58:30.486356 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.486291 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" podStartSLOduration=2.491934186 podStartE2EDuration="9.486271212s" totalImagesPullingTime="6.994337026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:58:30.483673341 +0000 UTC m=+1507.596318929" watchObservedRunningTime="2026-10-29 22:58:30.486271212 +0000 UTC m=+1507.598916809" Oct 29 22:58:30.706080 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:30.705647 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk"] Oct 29 22:58:30.706809 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:58:30.706783 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918e09c2_847f_4111_821b_36dae8b99089.slice/crio-d2ab95caf9cce1d9037c4520322dbe43ac5e756adc0a01b3bccf19b33d2d6735 WatchSource:0}: Error finding container d2ab95caf9cce1d9037c4520322dbe43ac5e756adc0a01b3bccf19b33d2d6735: Status 404 returned error can't find the container with id d2ab95caf9cce1d9037c4520322dbe43ac5e756adc0a01b3bccf19b33d2d6735 Oct 29 22:58:31.307464 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:31.307430 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" event={"ID":"918e09c2-847f-4111-821b-36dae8b99089","Type":"ContainerStarted","Data":"88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6"} Oct 29 22:58:31.307933 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:31.307491 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" event={"ID":"918e09c2-847f-4111-821b-36dae8b99089","Type":"ContainerStarted","Data":"d2ab95caf9cce1d9037c4520322dbe43ac5e756adc0a01b3bccf19b33d2d6735"} Oct 29 22:58:31.358339 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:31.358283 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" podStartSLOduration=1.358268963 podStartE2EDuration="1.358268963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:58:31.354670814 +0000 UTC m=+1508.467316402" watchObservedRunningTime="2026-10-29 22:58:31.358268963 +0000 UTC m=+1508.470914557" Oct 29 22:58:32.316401 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:32.316363 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:35.800919 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:58:35.800877 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:58:39.638499 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:39.638457 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:58:41.358865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:41.355049 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:58:41.358865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:41.356530 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:58:43.210863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:43.210019 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:58:43.210863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:43.210092 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:58:43.347864 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:43.341352 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:58:51.550894 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.550858 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4"] Oct 29 22:58:51.559422 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.559398 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.584382 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.584356 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 29 22:58:51.585379 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.585352 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4"] Oct 29 22:58:51.592413 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.592390 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wdpvc\"" Oct 29 22:58:51.684374 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.684345 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 29 22:58:51.684374 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.684345 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 29 22:58:51.718272 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718247 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.718416 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718284 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.718416 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718370 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.718416 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718403 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.718533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718453 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.718681 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718668 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.718725 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.718688 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ftz\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.819898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.819804 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.819898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.819859 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ftz\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.820135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.819961 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.820135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.819992 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.820135 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.820038 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.820300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.820170 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.820300 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.820229 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.820943 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.820919 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.822809 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.822781 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.823139 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.823114 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.823238 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.823157 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.823238 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.823227 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.841823 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.841795 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.842083 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.842064 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ftz\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dnq4\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:51.874207 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:51.874186 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:52.209999 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.209970 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4"] Oct 29 22:58:52.210436 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:58:52.210402 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2176fdbe_2a1d_4b30_9959_cc00cc909c84.slice/crio-aa3eec4e3d75dfe6d11c92c6ebd3a77e5d9a83326c98823e8b3e5bec43bfa2ea WatchSource:0}: Error finding container aa3eec4e3d75dfe6d11c92c6ebd3a77e5d9a83326c98823e8b3e5bec43bfa2ea: Status 404 returned error can't find the container with id aa3eec4e3d75dfe6d11c92c6ebd3a77e5d9a83326c98823e8b3e5bec43bfa2ea Oct 29 22:58:52.213421 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.213381 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:58:52.213501 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.213464 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:58:52.935295 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.935250 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" event={"ID":"2176fdbe-2a1d-4b30-9959-cc00cc909c84","Type":"ContainerStarted","Data":"188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec"} Oct 29 22:58:52.937099 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.937077 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:58:52.937569 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.937527 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" event={"ID":"2176fdbe-2a1d-4b30-9959-cc00cc909c84","Type":"ContainerStarted","Data":"aa3eec4e3d75dfe6d11c92c6ebd3a77e5d9a83326c98823e8b3e5bec43bfa2ea"} Oct 29 22:58:52.992110 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:52.992050 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" podStartSLOduration=1.992031385 podStartE2EDuration="1.992031385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:58:52.984981056 +0000 UTC m=+1530.097626645" watchObservedRunningTime="2026-10-29 22:58:52.992031385 +0000 UTC m=+1530.104676975" Oct 29 22:58:53.943932 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:58:53.943905 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 22:59:04.307494 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:04.307450 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:59:04.502863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:04.501025 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 22:59:04.548716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:04.546956 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:59:04.548716 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:04.547022 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:59:05.402865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:05.383235 5030 patch_prober.go:29] interesting pod/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 22:59:05.402865 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:05.383301 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 22:59:05.978862 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:05.978158 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:59:06.362916 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:06.334120 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded" Oct 29 22:59:10.683533 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.683114 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p"] Oct 29 22:59:10.688673 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.688649 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.717019 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.716994 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p"] Oct 29 22:59:10.777458 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.777428 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.777814 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.777782 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.777951 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.777825 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.778015 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.777991 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.778052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.778036 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttmv\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.778225 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.778207 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.778331 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.778313 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.778374 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.778360 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.778528 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.778512 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.879695 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.879660 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.879895 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.879706 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.879895 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.879762 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.879895 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.879825 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.879917 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.879940 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880024 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880074 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880055 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmv\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880296 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880136 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880296 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880169 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880296 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880182 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880296 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880228 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880629 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880601 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.880916 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.880896 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.882756 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.882733 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.882890 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.882819 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.901801 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.901771 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:10.901945 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:10.901930 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttmv\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv\") pod \"e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:11.009587 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.009093 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:11.356497 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.356468 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p"] Oct 29 22:59:11.358514 ip-10-0-136-228 kubenswrapper[5030]: W1029 22:59:11.358487 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea863520_7d1a_4d24_b491_e1b9cc180044.slice/crio-439d2319c185ae12fb9a0200c33c6f3e27f1fb57baa1e3b3088e8c97fa14c2bd WatchSource:0}: Error finding container 439d2319c185ae12fb9a0200c33c6f3e27f1fb57baa1e3b3088e8c97fa14c2bd: Status 404 returned error can't find the container with id 439d2319c185ae12fb9a0200c33c6f3e27f1fb57baa1e3b3088e8c97fa14c2bd Oct 29 22:59:11.361525 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.361485 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:59:11.361608 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.361574 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:59:11.361649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.361621 5030 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"} Oct 29 22:59:11.369402 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.369376 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz"] Oct 29 22:59:11.396893 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.396870 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624"] Oct 29 22:59:11.435304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:11.435236 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" event={"ID":"ea863520-7d1a-4d24-b491-e1b9cc180044","Type":"ContainerStarted","Data":"439d2319c185ae12fb9a0200c33c6f3e27f1fb57baa1e3b3088e8c97fa14c2bd"} Oct 29 22:59:12.444655 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:12.444620 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" event={"ID":"ea863520-7d1a-4d24-b491-e1b9cc180044","Type":"ContainerStarted","Data":"00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792"} Oct 29 22:59:12.557515 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:12.557433 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podStartSLOduration=2.55741245 podStartE2EDuration="2.55741245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 22:59:12.550915685 +0000 UTC m=+1549.663561275" watchObservedRunningTime="2026-10-29 22:59:12.55741245 +0000 UTC m=+1549.670058039" Oct 29 22:59:13.010103 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:13.010074 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:13.012342 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:13.012300 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 29 22:59:13.012501 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:13.012356 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 29 22:59:14.009778 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:14.009737 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 29 22:59:14.010218 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:14.009805 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 29 22:59:15.010578 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:15.010535 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 29 22:59:15.011012 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:15.010602 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 29 22:59:15.405567 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:15.405482 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:59:16.014897 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.009755 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 29 22:59:16.014897 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.009813 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 29 22:59:16.373212 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.373087 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" podUID="9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" containerName="payload-processing" containerID="cri-o://af54dfea1cc708b227d09ff5ed1c00d1e300f960f0dca24114e039a8ab999448" gracePeriod=25 Oct 29 22:59:16.397511 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.397298 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" podUID="a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" containerName="payload-pre-processing" containerID="cri-o://738bef8b0b514a38cbb65e5a6d533b23ef544dc796d277b51d718ae0de0b16d5" gracePeriod=25 Oct 29 22:59:16.588863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.583269 5030 generic.go:356] "Generic (PLEG): container finished" podID="a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" containerID="738bef8b0b514a38cbb65e5a6d533b23ef544dc796d277b51d718ae0de0b16d5" exitCode=0 Oct 29 22:59:16.588863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.583477 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" event={"ID":"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee","Type":"ContainerDied","Data":"738bef8b0b514a38cbb65e5a6d533b23ef544dc796d277b51d718ae0de0b16d5"} Oct 29 22:59:16.588863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.585988 5030 generic.go:356] "Generic (PLEG): container finished" podID="9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" containerID="af54dfea1cc708b227d09ff5ed1c00d1e300f960f0dca24114e039a8ab999448" exitCode=0 Oct 29 22:59:16.588863 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.586075 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" event={"ID":"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c","Type":"ContainerDied","Data":"af54dfea1cc708b227d09ff5ed1c00d1e300f960f0dca24114e039a8ab999448"} Oct 29 22:59:16.893077 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.892989 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:59:16.910744 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.910716 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:59:16.947380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.945868 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume\") pod \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " Oct 29 22:59:16.947380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.945947 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b\" (UniqueName: \"kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b\") pod \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " Oct 29 22:59:16.947380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.945973 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume\") pod \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\" (UID: \"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee\") " Oct 29 22:59:16.947380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.946017 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz\" (UniqueName: \"kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz\") pod \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\" (UID: \"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c\") " Oct 29 22:59:16.947380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.947266 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume" pod "a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" (UID: "a3e1fc4d-1753-42a7-9351-fc1d6695b7ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:16.947380 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.947365 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume" pod "9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" (UID: "9aa5f038-4ebd-45c4-b2f5-7074221c8d9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:16.948694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.948673 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz" pod "9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" (UID: "9aa5f038-4ebd-45c4-b2f5-7074221c8d9c"). InnerVolumeSpecName "kube-api-access-vzzbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:16.949052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:16.948984 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b" pod "a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" (UID: "a3e1fc4d-1753-42a7-9351-fc1d6695b7ee"). InnerVolumeSpecName "kube-api-access-nmj6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:17.160255 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.149402 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmj6b\" (UniqueName: \"kubernetes.io/projected/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-kube-api-access-nmj6b\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:17.160255 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.149437 5030 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee-plugins-config-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:17.160255 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.149453 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzzbz\" (UniqueName: \"kubernetes.io/projected/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-kube-api-access-vzzbz\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:17.160255 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.149466 5030 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c-plugins-config-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:17.193285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.191984 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 29 22:59:17.193285 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.192051 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 29 22:59:17.671630 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.671594 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" event={"ID":"a3e1fc4d-1753-42a7-9351-fc1d6695b7ee","Type":"ContainerDied","Data":"5f9a299a07509f4fdd437239369ee4c3555e29f3a00e713efd7cf98b9591ae0e"} Oct 29 22:59:17.671967 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.671947 5030 scope.go:122] "RemoveContainer" containerID="738bef8b0b514a38cbb65e5a6d533b23ef544dc796d277b51d718ae0de0b16d5" Oct 29 22:59:17.672261 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.672246 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624" Oct 29 22:59:17.831374 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.826161 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" event={"ID":"9aa5f038-4ebd-45c4-b2f5-7074221c8d9c","Type":"ContainerDied","Data":"729ce4ade7e9878f4e5f6838cac5da396e203517c0103f4e29a618a93b0d21a3"} Oct 29 22:59:17.831374 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.826282 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz" Oct 29 22:59:17.865001 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.864954 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624"] Oct 29 22:59:17.865445 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.865423 5030 scope.go:122] "RemoveContainer" containerID="af54dfea1cc708b227d09ff5ed1c00d1e300f960f0dca24114e039a8ab999448" Oct 29 22:59:17.903027 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.897735 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2bd3f07e-6bfd596755-qd624"] Oct 29 22:59:17.937573 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.937539 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz"] Oct 29 22:59:17.951094 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:17.951064 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2bd3f07e-75778c5d95-pg2pz"] Oct 29 22:59:18.011006 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:18.010971 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 29 22:59:18.011317 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:18.011285 5030 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 29 22:59:19.056176 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:19.056143 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" path="/var/lib/kubelet/pods/9aa5f038-4ebd-45c4-b2f5-7074221c8d9c/volumes" Oct 29 22:59:19.056898 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:19.056871 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" path="/var/lib/kubelet/pods/a3e1fc4d-1753-42a7-9351-fc1d6695b7ee/volumes" Oct 29 22:59:19.116457 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:19.116423 5030 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:19.118045 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:19.118025 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:19.119185 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:19.119163 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 22:59:24.749976 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:24.749943 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk"] Oct 29 22:59:28.542550 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:28.542277 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s"] Oct 29 22:59:28.543067 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:28.542542 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" podUID="d5048ca2-d798-4454-aa96-a4e0f180dd1d" containerName="maas-api" containerID="cri-o://0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c" gracePeriod=30 Oct 29 22:59:29.029257 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.028537 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:59:29.119132 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.119093 5030 generic.go:356] "Generic (PLEG): container finished" podID="d5048ca2-d798-4454-aa96-a4e0f180dd1d" containerID="0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c" exitCode=0 Oct 29 22:59:29.119281 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.119255 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" event={"ID":"d5048ca2-d798-4454-aa96-a4e0f180dd1d","Type":"ContainerDied","Data":"0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c"} Oct 29 22:59:29.119336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.119304 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" event={"ID":"d5048ca2-d798-4454-aa96-a4e0f180dd1d","Type":"ContainerDied","Data":"6ed48e177ac5d882c1c9fb98848e23c3307606f36128cfebad0d5532849c8204"} Oct 29 22:59:29.119336 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.119332 5030 scope.go:122] "RemoveContainer" containerID="0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c" Oct 29 22:59:29.119509 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.119496 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s" Oct 29 22:59:29.139127 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.139100 5030 scope.go:122] "RemoveContainer" containerID="0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c" Oct 29 22:59:29.139530 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:29.139499 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c\": container with ID starting with 0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c not found: ID does not exist" containerID="0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c" Oct 29 22:59:29.139628 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.139541 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c"} err="failed to get container status \"0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c\": rpc error: code = NotFound desc = could not find container \"0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c\": container with ID starting with 0c0d99ac26512c00c5b681408b3c89485e5ef141d959c90b5a7fe767a4e5af2c not found: ID does not exist" Oct 29 22:59:29.143414 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.143386 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls\") pod \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " Oct 29 22:59:29.143544 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.143497 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs\") pod \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " Oct 29 22:59:29.143660 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.143644 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s\" (UniqueName: \"kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s\") pod \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " Oct 29 22:59:29.143749 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.143687 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca\") pod \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\" (UID: \"d5048ca2-d798-4454-aa96-a4e0f180dd1d\") " Oct 29 22:59:29.147651 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.147619 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca" pod "d5048ca2-d798-4454-aa96-a4e0f180dd1d" (UID: "d5048ca2-d798-4454-aa96-a4e0f180dd1d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:29.284870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.254271 5030 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5048ca2-d798-4454-aa96-a4e0f180dd1d-openshift-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:29.284870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.270028 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls" pod "d5048ca2-d798-4454-aa96-a4e0f180dd1d" (UID: "d5048ca2-d798-4454-aa96-a4e0f180dd1d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:59:29.284870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.270162 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s" pod "d5048ca2-d798-4454-aa96-a4e0f180dd1d" (UID: "d5048ca2-d798-4454-aa96-a4e0f180dd1d"). InnerVolumeSpecName "kube-api-access-6s57s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:29.284870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.273958 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs" pod "d5048ca2-d798-4454-aa96-a4e0f180dd1d" (UID: "d5048ca2-d798-4454-aa96-a4e0f180dd1d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 22:59:29.356200 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.356167 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-maas-api-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:29.356200 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.356196 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5048ca2-d798-4454-aa96-a4e0f180dd1d-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:29.356200 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.356205 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s57s\" (UniqueName: \"kubernetes.io/projected/d5048ca2-d798-4454-aa96-a4e0f180dd1d-kube-api-access-6s57s\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:29.478723 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.478692 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s"] Oct 29 22:59:29.490496 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.490454 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dd3ff3-7776f6885d-lwr9s"] Oct 29 22:59:29.750783 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:29.750690 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" podUID="918e09c2-847f-4111-821b-36dae8b99089" containerName="payload-pre-processing" containerID="cri-o://88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6" gracePeriod=25 Oct 29 22:59:30.068123 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.068092 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:59:30.130456 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.130414 5030 generic.go:356] "Generic (PLEG): container finished" podID="918e09c2-847f-4111-821b-36dae8b99089" containerID="88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6" exitCode=0 Oct 29 22:59:30.130619 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.130580 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" Oct 29 22:59:30.131067 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.131037 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" event={"ID":"918e09c2-847f-4111-821b-36dae8b99089","Type":"ContainerDied","Data":"88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6"} Oct 29 22:59:30.131190 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.131134 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk" event={"ID":"918e09c2-847f-4111-821b-36dae8b99089","Type":"ContainerDied","Data":"d2ab95caf9cce1d9037c4520322dbe43ac5e756adc0a01b3bccf19b33d2d6735"} Oct 29 22:59:30.131190 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.131167 5030 scope.go:122] "RemoveContainer" containerID="88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6" Oct 29 22:59:30.145046 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.145024 5030 scope.go:122] "RemoveContainer" containerID="88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6" Oct 29 22:59:30.145374 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:30.145353 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6\": container with ID starting with 88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6 not found: ID does not exist" containerID="88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6" Oct 29 22:59:30.145459 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.145381 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6"} err="failed to get container status \"88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6\": rpc error: code = NotFound desc = could not find container \"88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6\": container with ID starting with 88ef70ef8a817260b111991879a591f2e02b950fce8854cb9a1daf089a71afe6 not found: ID does not exist" Oct 29 22:59:30.173862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.169605 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume\") pod \"918e09c2-847f-4111-821b-36dae8b99089\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " Oct 29 22:59:30.173862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.169663 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc\" (UniqueName: \"kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc\") pod \"918e09c2-847f-4111-821b-36dae8b99089\" (UID: \"918e09c2-847f-4111-821b-36dae8b99089\") " Oct 29 22:59:30.173862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.171220 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume" pod "918e09c2-847f-4111-821b-36dae8b99089" (UID: "918e09c2-847f-4111-821b-36dae8b99089"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:30.173862 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.173213 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc" pod "918e09c2-847f-4111-821b-36dae8b99089" (UID: "918e09c2-847f-4111-821b-36dae8b99089"). InnerVolumeSpecName "kube-api-access-hx4rc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:30.272064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.271993 5030 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918e09c2-847f-4111-821b-36dae8b99089-plugins-config-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:30.272064 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.272017 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx4rc\" (UniqueName: \"kubernetes.io/projected/918e09c2-847f-4111-821b-36dae8b99089-kube-api-access-hx4rc\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:30.483592 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.483559 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk"] Oct 29 22:59:30.491368 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:30.491340 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-01e33b3d-5d784b64c5-mqfsk"] Oct 29 22:59:31.046900 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:31.046869 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="918e09c2-847f-4111-821b-36dae8b99089" path="/var/lib/kubelet/pods/918e09c2-847f-4111-821b-36dae8b99089/volumes" Oct 29 22:59:31.047410 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:31.047395 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5048ca2-d798-4454-aa96-a4e0f180dd1d" path="/var/lib/kubelet/pods/d5048ca2-d798-4454-aa96-a4e0f180dd1d/volumes" Oct 29 22:59:35.831013 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:35.830591 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 22:59:36.955669 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:36.955634 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv"] Oct 29 22:59:36.956304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:36.955715 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq"] Oct 29 22:59:36.956304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:36.955945 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" containerID="cri-o://0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2" gracePeriod=30 Oct 29 22:59:36.956304 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:36.956206 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" containerID="cri-o://36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668" gracePeriod=30 Oct 29 22:59:36.956667 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:36.956641 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5"] Oct 29 22:59:36.959879 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:36.959827 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" containerID="cri-o://ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020" gracePeriod=30 Oct 29 22:59:42.816007 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.815772 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:59:42.844546 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.844516 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858182 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858248 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858282 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858315 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858353 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858406 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858445 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858511 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858583 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858611 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858690 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858721 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858784 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858853 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858887 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.858925 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.859001 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs\") pod \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\" (UID: \"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b\") " Oct 29 22:59:42.859820 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.859043 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data\") pod \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\" (UID: \"bdbf9d9f-ffa9-4969-a67c-29d728b914ec\") " Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.871227 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.871471 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.872059 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.872282 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874061 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874301 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874380 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874522 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874539 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874552 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.874795 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.874162 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:42.875563 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.875513 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 22:59:42.879950 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.875701 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.881680 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.881487 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "kube-api-access-g6nxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:42.882670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.882634 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "kube-api-access-rrrcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:42.889963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.889883 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token" pod "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" (UID: "d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:42.889963 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.890014 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 22:59:42.891622 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.891596 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy" pod "bdbf9d9f-ffa9-4969-a67c-29d728b914ec" (UID: "bdbf9d9f-ffa9-4969-a67c-29d728b914ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.936980 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.936949 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:59:42.960954 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.960925 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961161 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961140 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961233 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961186 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.961233 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961200 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961366 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961253 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961426 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961362 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961426 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961398 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961539 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961449 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.961539 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961481 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961546 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961649 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961581 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert\") pod \"34f977dc-3582-4a2d-b706-4dfd573a4863\" (UID: \"34f977dc-3582-4a2d-b706-4dfd573a4863\") " Oct 29 22:59:42.961808 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961782 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.961944 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962241 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962257 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962268 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrrcj\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-kube-api-access-rrrcj\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962277 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962286 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962294 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962301 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962309 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962317 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962324 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962332 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962339 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962347 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962354 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962362 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962369 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962377 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962384 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdbf9d9f-ffa9-4969-a67c-29d728b914ec-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962391 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962398 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962407 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.962431 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.962414 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6nxc\" (UniqueName: \"kubernetes.io/projected/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b-kube-api-access-g6nxc\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:42.964052 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.963022 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 22:59:42.964937 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.964906 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 22:59:42.966442 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.966416 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:42.971268 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.971241 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 22:59:42.980387 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:42.980348 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk" pod "34f977dc-3582-4a2d-b706-4dfd573a4863" (UID: "34f977dc-3582-4a2d-b706-4dfd573a4863"). InnerVolumeSpecName "kube-api-access-tmqfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 22:59:43.064694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.064025 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:43.064694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.064059 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmqfk\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-kube-api-access-tmqfk\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:43.064694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.064074 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:43.064694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.064087 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f977dc-3582-4a2d-b706-4dfd573a4863-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:43.064694 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.064099 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f977dc-3582-4a2d-b706-4dfd573a4863-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 22:59:43.486847 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.486808 5030 generic.go:356] "Generic (PLEG): container finished" podID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerID="ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020" exitCode=0 Oct 29 22:59:43.487049 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.486919 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" Oct 29 22:59:43.487133 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.486911 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" event={"ID":"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b","Type":"ContainerDied","Data":"ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020"} Oct 29 22:59:43.487133 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.487114 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5" event={"ID":"d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b","Type":"ContainerDied","Data":"6c2dfaa13a11be029c7dafc4d881d2347be0bccfe2ac0384e11f0829300e052c"} Oct 29 22:59:43.487288 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.487159 5030 scope.go:122] "RemoveContainer" containerID="ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020" Oct 29 22:59:43.490670 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.490646 5030 generic.go:356] "Generic (PLEG): container finished" podID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerID="36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668" exitCode=0 Oct 29 22:59:43.490793 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.490762 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" event={"ID":"34f977dc-3582-4a2d-b706-4dfd573a4863","Type":"ContainerDied","Data":"36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668"} Oct 29 22:59:43.490875 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.490807 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" event={"ID":"34f977dc-3582-4a2d-b706-4dfd573a4863","Type":"ContainerDied","Data":"ed496f9b4bfee76b0af85bf6d1025331fd9f65453814ac3cf04caf70456d339a"} Oct 29 22:59:43.490960 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.490894 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv" Oct 29 22:59:43.510198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.509332 5030 generic.go:356] "Generic (PLEG): container finished" podID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerID="0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2" exitCode=0 Oct 29 22:59:43.510198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.509500 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" event={"ID":"bdbf9d9f-ffa9-4969-a67c-29d728b914ec","Type":"ContainerDied","Data":"0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2"} Oct 29 22:59:43.510198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.509527 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" Oct 29 22:59:43.510198 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.509586 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq" event={"ID":"bdbf9d9f-ffa9-4969-a67c-29d728b914ec","Type":"ContainerDied","Data":"15bd0a85a212dc4dea8078eb2fcb5af3b816b8464cebe0a3477122616f679537"} Oct 29 22:59:43.516313 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.516226 5030 scope.go:122] "RemoveContainer" containerID="ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020" Oct 29 22:59:43.516553 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:43.516507 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020\": container with ID starting with ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020 not found: ID does not exist" containerID="ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020" Oct 29 22:59:43.516665 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.516542 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020"} err="failed to get container status \"ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020\": rpc error: code = NotFound desc = could not find container \"ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020\": container with ID starting with ff6bad064a9249171c83172f7d5e4721adfd100fdaaf0d01f9328f22f774e020 not found: ID does not exist" Oct 29 22:59:43.516665 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.516566 5030 scope.go:122] "RemoveContainer" containerID="36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668" Oct 29 22:59:43.537791 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.537770 5030 scope.go:122] "RemoveContainer" containerID="36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668" Oct 29 22:59:43.538128 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:43.538107 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668\": container with ID starting with 36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668 not found: ID does not exist" containerID="36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668" Oct 29 22:59:43.538242 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.538140 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668"} err="failed to get container status \"36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668\": rpc error: code = NotFound desc = could not find container \"36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668\": container with ID starting with 36bdf46f10e889d8d2e52b254ab9e8309b127bec2bf67361a369f9e9894ab668 not found: ID does not exist" Oct 29 22:59:43.538242 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.538161 5030 scope.go:122] "RemoveContainer" containerID="0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2" Oct 29 22:59:43.554756 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.554739 5030 scope.go:122] "RemoveContainer" containerID="0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2" Oct 29 22:59:43.555122 ip-10-0-136-228 kubenswrapper[5030]: E1029 22:59:43.555060 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2\": container with ID starting with 0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2 not found: ID does not exist" containerID="0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2" Oct 29 22:59:43.555122 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.555094 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2"} err="failed to get container status \"0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2\": rpc error: code = NotFound desc = could not find container \"0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2\": container with ID starting with 0449cf3f209ba161bdaed32b1900face2f8f64379be38b1f8172d25fc708b2d2 not found: ID does not exist" Oct 29 22:59:43.558213 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.558194 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv"] Oct 29 22:59:43.567050 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.567001 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-6lgqv"] Oct 29 22:59:43.595197 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.595150 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5"] Oct 29 22:59:43.598530 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.598511 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-gk2g5"] Oct 29 22:59:43.637053 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.636978 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq"] Oct 29 22:59:43.644870 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:43.644832 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-dd3ff3-openshift-default-74ffbb64df-h9xbq"] Oct 29 22:59:45.054653 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:45.054616 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" path="/var/lib/kubelet/pods/34f977dc-3582-4a2d-b706-4dfd573a4863/volumes" Oct 29 22:59:45.055161 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:45.055112 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" path="/var/lib/kubelet/pods/bdbf9d9f-ffa9-4969-a67c-29d728b914ec/volumes" Oct 29 22:59:45.055581 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:45.055563 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" path="/var/lib/kubelet/pods/d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b/volumes" Oct 29 22:59:55.719859 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:55.719722 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 22:59:56.154425 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:56.154330 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9"] Oct 29 22:59:56.154606 ip-10-0-136-228 kubenswrapper[5030]: I1029 22:59:56.154520 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" containerID="cri-o://43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9" gracePeriod=30 Oct 29 23:00:00.207912 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.207877 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk"] Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.213999 5030 cpu_manager.go:414] "containerMap: removing container" podUID="d5048ca2-d798-4454-aa96-a4e0f180dd1d" containerName="maas-api" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214025 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5048ca2-d798-4454-aa96-a4e0f180dd1d" containerName="maas-api" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214081 5030 cpu_manager.go:414] "containerMap: removing container" podUID="a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" containerName="payload-pre-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214089 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" containerName="payload-pre-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214112 5030 cpu_manager.go:414] "containerMap: removing container" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214120 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214140 5030 cpu_manager.go:414] "containerMap: removing container" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214147 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214158 5030 cpu_manager.go:414] "containerMap: removing container" podUID="918e09c2-847f-4111-821b-36dae8b99089" containerName="payload-pre-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214165 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="918e09c2-847f-4111-821b-36dae8b99089" containerName="payload-pre-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214192 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" containerName="payload-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214199 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" containerName="payload-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214224 5030 cpu_manager.go:414] "containerMap: removing container" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214231 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214412 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7a54399-f2a4-4e8d-86e6-c8acc9b4f90b" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214429 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="34f977dc-3582-4a2d-b706-4dfd573a4863" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214439 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="918e09c2-847f-4111-821b-36dae8b99089" containerName="payload-pre-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214454 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9aa5f038-4ebd-45c4-b2f5-7074221c8d9c" containerName="payload-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214466 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdbf9d9f-ffa9-4969-a67c-29d728b914ec" containerName="istio-proxy" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214479 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5048ca2-d798-4454-aa96-a4e0f180dd1d" containerName="maas-api" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.214495 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3e1fc4d-1753-42a7-9351-fc1d6695b7ee" containerName="payload-pre-processing" Oct 29 23:00:00.229127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.221017 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.249349 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.249325 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk"] Oct 29 23:00:00.293750 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.293719 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjz6s\"" Oct 29 23:00:00.326005 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.325971 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72pq7\" (UniqueName: \"kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7\") pod \"maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.326155 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.326039 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.428086 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.428052 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72pq7\" (UniqueName: \"kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7\") pod \"maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.428269 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.428122 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.428783 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.428761 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.450580 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.450551 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72pq7\" (UniqueName: \"kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7\") pod \"maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.550781 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.550753 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:00.867342 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:00.867307 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk"] Oct 29 23:00:00.873338 ip-10-0-136-228 kubenswrapper[5030]: W1029 23:00:00.873304 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f361422_5141_4f60_833a_e0ae904c5a70.slice/crio-686218fafdf28fd2723fda6d1c6f55b4d626af013dc8b28e4ab653753cddbb6e WatchSource:0}: Error finding container 686218fafdf28fd2723fda6d1c6f55b4d626af013dc8b28e4ab653753cddbb6e: Status 404 returned error can't find the container with id 686218fafdf28fd2723fda6d1c6f55b4d626af013dc8b28e4ab653753cddbb6e Oct 29 23:00:01.424040 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.424013 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 23:00:01.548532 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548504 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.548728 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548544 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.548728 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548593 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.548728 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548688 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.548922 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548833 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.548922 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548832 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:01.548922 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548889 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.549070 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.548943 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.549070 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.549006 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:01.549169 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.549086 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.549169 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.549141 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy\") pod \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\" (UID: \"344cdc2c-85cc-4eb5-bb00-523b63b26b2c\") " Oct 29 23:00:01.549281 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.549207 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:00:01.549992 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.549963 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:01.550203 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.550178 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:01.550347 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.550328 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.550433 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.550352 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.550433 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.550369 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.550433 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.550382 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.550433 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.550395 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.552410 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.552382 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:01.553992 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.553968 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:01.553992 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.553981 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:00:01.554148 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.554006 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg" pod "344cdc2c-85cc-4eb5-bb00-523b63b26b2c" (UID: "344cdc2c-85cc-4eb5-bb00-523b63b26b2c"). InnerVolumeSpecName "kube-api-access-6wwvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:01.651900 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.651871 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.651900 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.651896 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wwvg\" (UniqueName: \"kubernetes.io/projected/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-kube-api-access-6wwvg\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.651900 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.651906 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.652149 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.651916 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/344cdc2c-85cc-4eb5-bb00-523b63b26b2c-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:01.678551 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.678472 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" event={"ID":"9f361422-5141-4f60-833a-e0ae904c5a70","Type":"ContainerStarted","Data":"686218fafdf28fd2723fda6d1c6f55b4d626af013dc8b28e4ab653753cddbb6e"} Oct 29 23:00:01.680067 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.680037 5030 generic.go:356] "Generic (PLEG): container finished" podID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerID="43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9" exitCode=0 Oct 29 23:00:01.680220 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.680094 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" Oct 29 23:00:01.680220 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.680127 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" event={"ID":"344cdc2c-85cc-4eb5-bb00-523b63b26b2c","Type":"ContainerDied","Data":"43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9"} Oct 29 23:00:01.680220 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.680174 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9" event={"ID":"344cdc2c-85cc-4eb5-bb00-523b63b26b2c","Type":"ContainerDied","Data":"ac994be78d9a2646e86425ebe20991b310da3d52d486a78518fb6e7646a0a655"} Oct 29 23:00:01.680220 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.680202 5030 scope.go:122] "RemoveContainer" containerID="43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9" Oct 29 23:00:01.692165 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.692146 5030 scope.go:122] "RemoveContainer" containerID="43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9" Oct 29 23:00:01.692470 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:01.692452 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9\": container with ID starting with 43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9 not found: ID does not exist" containerID="43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9" Oct 29 23:00:01.692539 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.692478 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9"} err="failed to get container status \"43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9\": rpc error: code = NotFound desc = could not find container \"43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9\": container with ID starting with 43732ce0d9349de2fc9f85be9c6439c90aa94257277f7abef5905dcfa01077c9 not found: ID does not exist" Oct 29 23:00:01.748366 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.748329 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9"] Oct 29 23:00:01.754977 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:01.754953 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-5bf55a-openshift-default-84668b5c9-lphs9"] Oct 29 23:00:03.054275 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:03.054241 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" path="/var/lib/kubelet/pods/344cdc2c-85cc-4eb5-bb00-523b63b26b2c/volumes" Oct 29 23:00:04.709062 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:04.709035 5030 generic.go:356] "Generic (PLEG): container finished" podID="9f361422-5141-4f60-833a-e0ae904c5a70" containerID="51cdc0f0fc1f1d8bc3663878531bf69946396f8541bc4d75d5627d75d8ab4b4d" exitCode=60 Oct 29 23:00:04.709413 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:04.709118 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" event={"ID":"9f361422-5141-4f60-833a-e0ae904c5a70","Type":"ContainerDied","Data":"51cdc0f0fc1f1d8bc3663878531bf69946396f8541bc4d75d5627d75d8ab4b4d"} Oct 29 23:00:04.709413 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:04.709358 5030 scope.go:122] "RemoveContainer" containerID="51cdc0f0fc1f1d8bc3663878531bf69946396f8541bc4d75d5627d75d8ab4b4d" Oct 29 23:00:05.719602 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:05.719569 5030 generic.go:356] "Generic (PLEG): container finished" podID="9f361422-5141-4f60-833a-e0ae904c5a70" containerID="c8a7eadddf6c34ba490b6a2d07b02a4938f0c9d026200409bbca98eef5e5c043" exitCode=60 Oct 29 23:00:05.720119 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:05.719652 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" event={"ID":"9f361422-5141-4f60-833a-e0ae904c5a70","Type":"ContainerDied","Data":"c8a7eadddf6c34ba490b6a2d07b02a4938f0c9d026200409bbca98eef5e5c043"} Oct 29 23:00:05.720119 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:05.719712 5030 scope.go:122] "RemoveContainer" containerID="51cdc0f0fc1f1d8bc3663878531bf69946396f8541bc4d75d5627d75d8ab4b4d" Oct 29 23:00:05.720119 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:05.720052 5030 scope.go:122] "RemoveContainer" containerID="c8a7eadddf6c34ba490b6a2d07b02a4938f0c9d026200409bbca98eef5e5c043" Oct 29 23:00:05.720422 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:05.720399 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk_odh-ai-gateway-infra(9f361422-5141-4f60-833a-e0ae904c5a70)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" Oct 29 23:00:05.804464 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:05.804424 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:00:06.729727 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:06.729700 5030 scope.go:122] "RemoveContainer" containerID="c8a7eadddf6c34ba490b6a2d07b02a4938f0c9d026200409bbca98eef5e5c043" Oct 29 23:00:06.730155 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:06.730030 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk_odh-ai-gateway-infra(9f361422-5141-4f60-833a-e0ae904c5a70)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" Oct 29 23:00:19.038910 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:19.038878 5030 scope.go:122] "RemoveContainer" containerID="c8a7eadddf6c34ba490b6a2d07b02a4938f0c9d026200409bbca98eef5e5c043" Oct 29 23:00:19.841168 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:19.841134 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" event={"ID":"9f361422-5141-4f60-833a-e0ae904c5a70","Type":"ContainerStarted","Data":"82646e0f6ef8f05d258faa77b9cf75e12e5227a0f3e1cc2fd4cb047aa872321c"} Oct 29 23:00:19.890825 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:19.890775 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" podStartSLOduration=17.203856768 podStartE2EDuration="19.890760076s" totalImagesPullingTime="2.686903308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:00:19.888930855 +0000 UTC m=+1617.001576444" watchObservedRunningTime="2026-10-29 23:00:19.890760076 +0000 UTC m=+1617.003405663" Oct 29 23:00:20.850644 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:20.850614 5030 generic.go:356] "Generic (PLEG): container finished" podID="9f361422-5141-4f60-833a-e0ae904c5a70" containerID="82646e0f6ef8f05d258faa77b9cf75e12e5227a0f3e1cc2fd4cb047aa872321c" exitCode=60 Oct 29 23:00:20.851167 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:20.850691 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" event={"ID":"9f361422-5141-4f60-833a-e0ae904c5a70","Type":"ContainerDied","Data":"82646e0f6ef8f05d258faa77b9cf75e12e5227a0f3e1cc2fd4cb047aa872321c"} Oct 29 23:00:20.851167 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:20.850753 5030 scope.go:122] "RemoveContainer" containerID="c8a7eadddf6c34ba490b6a2d07b02a4938f0c9d026200409bbca98eef5e5c043" Oct 29 23:00:20.851300 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:20.851177 5030 scope.go:122] "RemoveContainer" containerID="82646e0f6ef8f05d258faa77b9cf75e12e5227a0f3e1cc2fd4cb047aa872321c" Oct 29 23:00:20.851523 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:20.851502 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk_odh-ai-gateway-infra(9f361422-5141-4f60-833a-e0ae904c5a70)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" Oct 29 23:00:20.908194 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:20.908164 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk"] Oct 29 23:00:22.166943 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.166871 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:22.188614 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.188591 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca\") pod \"9f361422-5141-4f60-833a-e0ae904c5a70\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " Oct 29 23:00:22.188748 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.188688 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7\" (UniqueName: \"kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7\") pod \"9f361422-5141-4f60-833a-e0ae904c5a70\" (UID: \"9f361422-5141-4f60-833a-e0ae904c5a70\") " Oct 29 23:00:22.189031 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.188972 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca" pod "9f361422-5141-4f60-833a-e0ae904c5a70" (UID: "9f361422-5141-4f60-833a-e0ae904c5a70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:00:22.189284 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.189255 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f361422-5141-4f60-833a-e0ae904c5a70-maas-api-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:22.191088 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.191067 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7" pod "9f361422-5141-4f60-833a-e0ae904c5a70" (UID: "9f361422-5141-4f60-833a-e0ae904c5a70"). InnerVolumeSpecName "kube-api-access-72pq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:22.291179 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.291148 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72pq7\" (UniqueName: \"kubernetes.io/projected/9f361422-5141-4f60-833a-e0ae904c5a70-kube-api-access-72pq7\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:22.907562 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.907526 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" Oct 29 23:00:22.907774 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.907596 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk" event={"ID":"9f361422-5141-4f60-833a-e0ae904c5a70","Type":"ContainerDied","Data":"686218fafdf28fd2723fda6d1c6f55b4d626af013dc8b28e4ab653753cddbb6e"} Oct 29 23:00:22.907774 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.907670 5030 scope.go:122] "RemoveContainer" containerID="82646e0f6ef8f05d258faa77b9cf75e12e5227a0f3e1cc2fd4cb047aa872321c" Oct 29 23:00:22.968722 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.968695 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk"] Oct 29 23:00:22.982281 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:22.982254 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-5bf55a-29888580-sqtkk"] Oct 29 23:00:23.047962 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:23.047929 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" path="/var/lib/kubelet/pods/9f361422-5141-4f60-833a-e0ae904c5a70/volumes" Oct 29 23:00:23.351574 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:23.351538 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2"] Oct 29 23:00:29.369102 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:29.369067 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps"] Oct 29 23:00:30.531699 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:30.531666 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:00:35.863718 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:35.863677 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:00:38.377487 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:38.377442 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" containerID="cri-o://a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb" gracePeriod=45 Oct 29 23:00:38.914808 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:38.914785 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 23:00:39.008765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.007329 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc\" (UniqueName: \"kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc\") pod \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " Oct 29 23:00:39.008765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.007423 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir\") pod \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " Oct 29 23:00:39.008765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.007459 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home\") pod \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " Oct 29 23:00:39.008765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.007680 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm\") pod \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " Oct 29 23:00:39.008765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.007709 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache\") pod \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " Oct 29 23:00:39.008765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.007746 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs\") pod \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\" (UID: \"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82\") " Oct 29 23:00:39.009233 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.009034 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home" pod "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" (UID: "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:39.009718 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.009692 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir" pod "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" (UID: "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:39.010390 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.010241 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache" pod "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" (UID: "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:39.022242 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.022209 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs" pod "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" (UID: "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:39.022357 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.022297 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm" pod "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" (UID: "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:39.024053 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.024027 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc" pod "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" (UID: "e0661d7c-66fc-4a2b-8251-60f6b7bc0c82"). InnerVolumeSpecName "kube-api-access-mnsbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:39.067249 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.067218 5030 generic.go:356] "Generic (PLEG): container finished" podID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerID="a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb" exitCode=0 Oct 29 23:00:39.067409 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.067337 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" Oct 29 23:00:39.067472 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.067431 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" event={"ID":"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82","Type":"ContainerDied","Data":"a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb"} Oct 29 23:00:39.067530 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.067465 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2" event={"ID":"e0661d7c-66fc-4a2b-8251-60f6b7bc0c82","Type":"ContainerDied","Data":"4e93bb319bff1f98b82627ba22fc398f596a1fbe6a8359cf42f08985b4363a89"} Oct 29 23:00:39.067584 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.067531 5030 scope.go:122] "RemoveContainer" containerID="a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb" Oct 29 23:00:39.081972 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.081946 5030 scope.go:122] "RemoveContainer" containerID="a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb" Oct 29 23:00:39.082353 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:39.082328 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb\": container with ID starting with a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb not found: ID does not exist" containerID="a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb" Oct 29 23:00:39.082450 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.082365 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb"} err="failed to get container status \"a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb\": rpc error: code = NotFound desc = could not find container \"a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb\": container with ID starting with a28e7e39eb99a13aec8dd0b1cf30d47d670af027ec4ff68825e73b1f4aad59cb not found: ID does not exist" Oct 29 23:00:39.109598 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.109568 5030 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-home\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:39.109598 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.109598 5030 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-dshm\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:39.109747 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.109608 5030 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-model-cache\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:39.109747 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.109618 5030 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tls-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:39.109747 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.109627 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnsbc\" (UniqueName: \"kubernetes.io/projected/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-kube-api-access-mnsbc\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:39.109747 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.109636 5030 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82-tmp-dir\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:39.130623 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.130600 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2"] Oct 29 23:00:39.141781 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:39.141755 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-5bf55a/premium-w5-5bf55a-kserve-7fd9795857-q8xp2"] Oct 29 23:00:41.049048 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:41.049009 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" path="/var/lib/kubelet/pods/e0661d7c-66fc-4a2b-8251-60f6b7bc0c82/volumes" Oct 29 23:00:44.393672 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.393607 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" containerID="cri-o://a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3" gracePeriod=45 Oct 29 23:00:44.487426 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.487389 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.89:8000/ready\": dial tcp 10.132.0.89:8000: connect: connection refused" Oct 29 23:00:44.714698 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.714673 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 23:00:44.796378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.796341 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache\") pod \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " Oct 29 23:00:44.796556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.796445 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs\" (UniqueName: \"kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs\") pod \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " Oct 29 23:00:44.796556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.796519 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir\") pod \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " Oct 29 23:00:44.796763 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.796746 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm\") pod \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " Oct 29 23:00:44.797123 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.797104 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home\") pod \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " Oct 29 23:00:44.797206 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.797178 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n\" (UniqueName: \"kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n\") pod \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\" (UID: \"947f7f47-a7ce-419e-8b65-3f8c74cb608f\") " Oct 29 23:00:44.799362 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.799209 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache" pod "947f7f47-a7ce-419e-8b65-3f8c74cb608f" (UID: "947f7f47-a7ce-419e-8b65-3f8c74cb608f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:44.799498 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.799397 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home" pod "947f7f47-a7ce-419e-8b65-3f8c74cb608f" (UID: "947f7f47-a7ce-419e-8b65-3f8c74cb608f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:44.799954 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.799916 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir" pod "947f7f47-a7ce-419e-8b65-3f8c74cb608f" (UID: "947f7f47-a7ce-419e-8b65-3f8c74cb608f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:44.803895 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.803597 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm" pod "947f7f47-a7ce-419e-8b65-3f8c74cb608f" (UID: "947f7f47-a7ce-419e-8b65-3f8c74cb608f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:00:44.804326 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.804292 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs" pod "947f7f47-a7ce-419e-8b65-3f8c74cb608f" (UID: "947f7f47-a7ce-419e-8b65-3f8c74cb608f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:44.805105 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.805084 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n" pod "947f7f47-a7ce-419e-8b65-3f8c74cb608f" (UID: "947f7f47-a7ce-419e-8b65-3f8c74cb608f"). InnerVolumeSpecName "kube-api-access-ppz4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:44.904529 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.904492 5030 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-model-cache\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:44.904529 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.904527 5030 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tls-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:44.904529 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.904538 5030 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-tmp-dir\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:44.904912 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.904546 5030 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-dshm\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:44.904912 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.904556 5030 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/947f7f47-a7ce-419e-8b65-3f8c74cb608f-home\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:44.904912 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:44.904565 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppz4n\" (UniqueName: \"kubernetes.io/projected/947f7f47-a7ce-419e-8b65-3f8c74cb608f-kube-api-access-ppz4n\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:45.115100 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.115064 5030 generic.go:356] "Generic (PLEG): container finished" podID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerID="a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3" exitCode=0 Oct 29 23:00:45.115282 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.115107 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" Oct 29 23:00:45.115282 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.115116 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" event={"ID":"947f7f47-a7ce-419e-8b65-3f8c74cb608f","Type":"ContainerDied","Data":"a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3"} Oct 29 23:00:45.115282 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.115157 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps" event={"ID":"947f7f47-a7ce-419e-8b65-3f8c74cb608f","Type":"ContainerDied","Data":"eac594cd1acc9d4fabe34f9350d15a85cc653788c16a493a0deb5ce27ef9a189"} Oct 29 23:00:45.115282 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.115181 5030 scope.go:122] "RemoveContainer" containerID="a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3" Oct 29 23:00:45.130819 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.130794 5030 scope.go:122] "RemoveContainer" containerID="a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3" Oct 29 23:00:45.131218 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:45.131164 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3\": container with ID starting with a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3 not found: ID does not exist" containerID="a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3" Oct 29 23:00:45.131218 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.131197 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3"} err="failed to get container status \"a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3\": rpc error: code = NotFound desc = could not find container \"a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3\": container with ID starting with a0d8360174f1e604ee6f83695eacad4671fd85080b212ac6bf5624bc3467d8f3 not found: ID does not exist" Oct 29 23:00:45.182114 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.182079 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps"] Oct 29 23:00:45.191097 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:45.191073 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-a2e7c8/distinct-2-w2-a2e7c8-kserve-699fdf89b-q2zps"] Oct 29 23:00:47.050511 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:47.050469 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" path="/var/lib/kubelet/pods/947f7f47-a7ce-419e-8b65-3f8c74cb608f/volumes" Oct 29 23:00:50.146454 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:50.146378 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:00:53.542859 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:53.541029 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw"] Oct 29 23:00:53.542859 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:53.541279 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" podUID="193b43e0-1296-4296-b7af-001487290791" containerName="maas-api" containerID="cri-o://0ba12be35e2aa4b3569d25e2e3023feb631546402bbb3504868f8fcfce43e439" gracePeriod=30 Oct 29 23:00:54.101862 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.097145 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp"] Oct 29 23:00:54.101862 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.097410 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" podUID="51e10dc5-da26-495c-8e26-574a86c666b0" containerName="maas-api" containerID="cri-o://b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f" gracePeriod=30 Oct 29 23:00:54.227309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.227273 5030 generic.go:356] "Generic (PLEG): container finished" podID="193b43e0-1296-4296-b7af-001487290791" containerID="0ba12be35e2aa4b3569d25e2e3023feb631546402bbb3504868f8fcfce43e439" exitCode=0 Oct 29 23:00:54.227447 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.227401 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" event={"ID":"193b43e0-1296-4296-b7af-001487290791","Type":"ContainerDied","Data":"0ba12be35e2aa4b3569d25e2e3023feb631546402bbb3504868f8fcfce43e439"} Oct 29 23:00:54.342996 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.342970 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 23:00:54.366411 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.366375 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs\") pod \"193b43e0-1296-4296-b7af-001487290791\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " Oct 29 23:00:54.366592 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.366575 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls\") pod \"193b43e0-1296-4296-b7af-001487290791\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " Oct 29 23:00:54.366698 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.366682 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca\") pod \"193b43e0-1296-4296-b7af-001487290791\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " Oct 29 23:00:54.366792 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.366735 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq\" (UniqueName: \"kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq\") pod \"193b43e0-1296-4296-b7af-001487290791\" (UID: \"193b43e0-1296-4296-b7af-001487290791\") " Oct 29 23:00:54.374537 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.374289 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls" pod "193b43e0-1296-4296-b7af-001487290791" (UID: "193b43e0-1296-4296-b7af-001487290791"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:54.375550 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.374823 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca" pod "193b43e0-1296-4296-b7af-001487290791" (UID: "193b43e0-1296-4296-b7af-001487290791"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:00:54.389500 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.389448 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq" pod "193b43e0-1296-4296-b7af-001487290791" (UID: "193b43e0-1296-4296-b7af-001487290791"). InnerVolumeSpecName "kube-api-access-8g5gq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:54.411485 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.411448 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs" pod "193b43e0-1296-4296-b7af-001487290791" (UID: "193b43e0-1296-4296-b7af-001487290791"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:54.476516 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.472680 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.476516 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.472724 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/193b43e0-1296-4296-b7af-001487290791-maas-api-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.476516 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.472739 5030 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/193b43e0-1296-4296-b7af-001487290791-openshift-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.476516 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.472756 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g5gq\" (UniqueName: \"kubernetes.io/projected/193b43e0-1296-4296-b7af-001487290791-kube-api-access-8g5gq\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.677069 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.676496 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 23:00:54.698147 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.698022 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4\" (UniqueName: \"kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4\") pod \"51e10dc5-da26-495c-8e26-574a86c666b0\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " Oct 29 23:00:54.698147 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.698091 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca\") pod \"51e10dc5-da26-495c-8e26-574a86c666b0\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " Oct 29 23:00:54.698147 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.698133 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls\") pod \"51e10dc5-da26-495c-8e26-574a86c666b0\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " Oct 29 23:00:54.698404 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.698316 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs\") pod \"51e10dc5-da26-495c-8e26-574a86c666b0\" (UID: \"51e10dc5-da26-495c-8e26-574a86c666b0\") " Oct 29 23:00:54.700866 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.698938 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca" pod "51e10dc5-da26-495c-8e26-574a86c666b0" (UID: "51e10dc5-da26-495c-8e26-574a86c666b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:00:54.702196 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.702165 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4" pod "51e10dc5-da26-495c-8e26-574a86c666b0" (UID: "51e10dc5-da26-495c-8e26-574a86c666b0"). InnerVolumeSpecName "kube-api-access-k45f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:54.702362 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.702333 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls" pod "51e10dc5-da26-495c-8e26-574a86c666b0" (UID: "51e10dc5-da26-495c-8e26-574a86c666b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:54.703453 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.703436 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs" pod "51e10dc5-da26-495c-8e26-574a86c666b0" (UID: "51e10dc5-da26-495c-8e26-574a86c666b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:54.802831 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.801748 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k45f4\" (UniqueName: \"kubernetes.io/projected/51e10dc5-da26-495c-8e26-574a86c666b0-kube-api-access-k45f4\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.802831 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.801794 5030 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e10dc5-da26-495c-8e26-574a86c666b0-openshift-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.802831 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.801810 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-maas-api-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:54.802831 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:54.801823 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51e10dc5-da26-495c-8e26-574a86c666b0-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:55.236909 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.236875 5030 generic.go:356] "Generic (PLEG): container finished" podID="51e10dc5-da26-495c-8e26-574a86c666b0" containerID="b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f" exitCode=0 Oct 29 23:00:55.237087 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.236937 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" Oct 29 23:00:55.237087 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.236967 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" event={"ID":"51e10dc5-da26-495c-8e26-574a86c666b0","Type":"ContainerDied","Data":"b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f"} Oct 29 23:00:55.237087 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.237028 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp" event={"ID":"51e10dc5-da26-495c-8e26-574a86c666b0","Type":"ContainerDied","Data":"e2303a0576efcc287b234e89c7ba926d8f64069b0aea0cf236abd48ad10f10b8"} Oct 29 23:00:55.237087 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.237060 5030 scope.go:122] "RemoveContainer" containerID="b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f" Oct 29 23:00:55.238745 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.238722 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" Oct 29 23:00:55.238830 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.238742 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw" event={"ID":"193b43e0-1296-4296-b7af-001487290791","Type":"ContainerDied","Data":"130ced2277b6a40102999ea50162fe16e04bba50516d52272ecc586e69958f2c"} Oct 29 23:00:55.248391 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.248372 5030 scope.go:122] "RemoveContainer" containerID="b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f" Oct 29 23:00:55.248671 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:00:55.248651 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f\": container with ID starting with b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f not found: ID does not exist" containerID="b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f" Oct 29 23:00:55.248734 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.248680 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f"} err="failed to get container status \"b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f\": rpc error: code = NotFound desc = could not find container \"b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f\": container with ID starting with b00335061a04ec01d00c9918868ef2553fe2758cbbc3b4eb086a326defec120f not found: ID does not exist" Oct 29 23:00:55.248734 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.248705 5030 scope.go:122] "RemoveContainer" containerID="0ba12be35e2aa4b3569d25e2e3023feb631546402bbb3504868f8fcfce43e439" Oct 29 23:00:55.318176 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.318112 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp"] Oct 29 23:00:55.331815 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.331788 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a2e7c8-59b84bbf54-zrbzp"] Oct 29 23:00:55.369502 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.369467 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw"] Oct 29 23:00:55.378304 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:55.378274 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-81f92d-6db8456994-627pw"] Oct 29 23:00:56.950540 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:56.950507 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn"] Oct 29 23:00:56.951088 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:56.950736 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" podUID="265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" containerName="maas-api" containerID="cri-o://489b0935b63513783a162a4d9930f6703f8aad0b7bccc0f656b7e5c488dfc427" gracePeriod=30 Oct 29 23:00:57.136192 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.131768 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="193b43e0-1296-4296-b7af-001487290791" path="/var/lib/kubelet/pods/193b43e0-1296-4296-b7af-001487290791/volumes" Oct 29 23:00:57.136192 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.133681 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51e10dc5-da26-495c-8e26-574a86c666b0" path="/var/lib/kubelet/pods/51e10dc5-da26-495c-8e26-574a86c666b0/volumes" Oct 29 23:00:57.311994 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.309745 5030 generic.go:356] "Generic (PLEG): container finished" podID="265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" containerID="489b0935b63513783a162a4d9930f6703f8aad0b7bccc0f656b7e5c488dfc427" exitCode=0 Oct 29 23:00:57.311994 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.309791 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" event={"ID":"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273","Type":"ContainerDied","Data":"489b0935b63513783a162a4d9930f6703f8aad0b7bccc0f656b7e5c488dfc427"} Oct 29 23:00:57.518174 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.518148 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 23:00:57.715947 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.696770 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs\") pod \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " Oct 29 23:00:57.715947 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.696989 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls\") pod \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " Oct 29 23:00:57.715947 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.697029 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj\" (UniqueName: \"kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj\") pod \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " Oct 29 23:00:57.715947 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.697174 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca\") pod \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\" (UID: \"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273\") " Oct 29 23:00:57.715947 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.698081 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca" pod "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" (UID: "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:00:57.817871 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.817813 5030 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-openshift-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:58.009226 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:57.998222 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs" pod "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" (UID: "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:58.009946 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.009853 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj" pod "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" (UID: "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273"). InnerVolumeSpecName "kube-api-access-g7rtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:00:58.014863 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.014383 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls" pod "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" (UID: "265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:00:58.155509 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.155428 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-maas-api-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:58.155509 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.155464 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7rtj\" (UniqueName: \"kubernetes.io/projected/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-kube-api-access-g7rtj\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:58.155509 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.155480 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:00:58.364584 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.364152 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" Oct 29 23:00:58.365299 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.365250 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn" event={"ID":"265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273","Type":"ContainerDied","Data":"c1ad2d67b16200c8c5de097832d704ac797381f68a5226a261f66e9e26680942"} Oct 29 23:00:58.365509 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.365493 5030 scope.go:122] "RemoveContainer" containerID="489b0935b63513783a162a4d9930f6703f8aad0b7bccc0f656b7e5c488dfc427" Oct 29 23:00:58.597955 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.597884 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn"] Oct 29 23:00:58.655937 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:58.644590 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-20f88f-bc6c99455-xbptn"] Oct 29 23:00:59.157871 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:00:59.157535 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" path="/var/lib/kubelet/pods/265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273/volumes" Oct 29 23:01:05.803426 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:05.803391 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:01:05.805861 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:05.805818 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch"] Oct 29 23:01:05.806118 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:05.806089 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" containerID="cri-o://7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965" gracePeriod=30 Oct 29 23:01:05.817568 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:05.817545 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq"] Oct 29 23:01:05.817935 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:05.817906 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" containerID="cri-o://d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16" gracePeriod=30 Oct 29 23:01:05.881430 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:05.881400 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn"] Oct 29 23:01:05.881667 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:05.881642 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" containerID="cri-o://0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92" gracePeriod=30 Oct 29 23:01:09.429303 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:09.427145 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms"] Oct 29 23:01:09.429303 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:09.427385 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" containerID="cri-o://b8fac82d97f566103ca88767fd49eea451e255eb1c1a6b1d6ccdc59517cad63b" gracePeriod=30 Oct 29 23:01:09.448095 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:09.448057 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h"] Oct 29 23:01:09.448356 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:09.448313 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" containerID="cri-o://bdb6d1240d9f7cee759899fdd2888971052f3e2d06d2dbd3ce97a27da4b4f06a" gracePeriod=30 Oct 29 23:01:09.479083 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:09.479051 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq"] Oct 29 23:01:09.479247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:09.479228 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" containerID="cri-o://dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3" gracePeriod=30 Oct 29 23:01:11.129131 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.128887 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:01:11.129131 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.128947 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:01:11.224889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.217322 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 23:01:11.224889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.220453 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 23:01:11.290402 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.290368 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 23:01:11.374620 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374531 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.374620 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374574 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.374620 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374617 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374652 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374682 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374733 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374766 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374827 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374909 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374909 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.374982 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375073 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375029 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375098 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375151 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375212 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375251 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375299 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375394 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375425 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo\") pod \"f3423324-ad21-4998-a379-6baf63bd8f00\" (UID: \"f3423324-ad21-4998-a379-6baf63bd8f00\") " Oct 29 23:01:11.375533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.375492 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert\") pod \"191a1836-d335-4756-aa1e-fbc676effe2b\" (UID: \"191a1836-d335-4756-aa1e-fbc676effe2b\") " Oct 29 23:01:11.376340 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.376311 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:11.377020 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.376806 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.377020 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.376820 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.377524 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.377382 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.377788 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.377476 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.377788 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.377566 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.377958 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.377796 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378161 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378189 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378204 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378218 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378230 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378244 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378256 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3423324-ad21-4998-a379-6baf63bd8f00-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378267 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378278 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.378462 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.378433 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:11.379164 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.379139 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:11.384016 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.383989 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:11.387751 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.387713 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:11.389137 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.380117 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "kube-api-access-stp27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:11.390219 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.390199 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.390568 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.390328 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:11.390568 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.390411 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy" pod "f3423324-ad21-4998-a379-6baf63bd8f00" (UID: "f3423324-ad21-4998-a379-6baf63bd8f00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.391913 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.391882 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr" pod "191a1836-d335-4756-aa1e-fbc676effe2b" (UID: "191a1836-d335-4756-aa1e-fbc676effe2b"). InnerVolumeSpecName "kube-api-access-5vzdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:11.439060 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.439032 5030 patch_prober.go:29] interesting pod/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:01:11.439180 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.439084 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:01:11.489920 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.489883 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.489953 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490285 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490261 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.490432 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490411 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490521 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490454 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490521 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490501 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490627 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490530 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490627 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490597 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:11.490627 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490619 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490786 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490716 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.490786 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.490749 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token\") pod \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\" (UID: \"75e01f27-9df6-4157-b1d8-77a71ae0f62d\") " Oct 29 23:01:11.491336 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491136 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.491336 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491312 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.491494 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491423 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stp27\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-kube-api-access-stp27\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491494 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491444 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3423324-ad21-4998-a379-6baf63bd8f00-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491494 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491458 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491494 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491477 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491507 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vzdr\" (UniqueName: \"kubernetes.io/projected/191a1836-d335-4756-aa1e-fbc676effe2b-kube-api-access-5vzdr\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491522 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3423324-ad21-4998-a379-6baf63bd8f00-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491535 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/191a1836-d335-4756-aa1e-fbc676effe2b-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491546 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491558 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/191a1836-d335-4756-aa1e-fbc676effe2b-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491571 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491583 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3423324-ad21-4998-a379-6baf63bd8f00-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491600 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491611 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/191a1836-d335-4756-aa1e-fbc676effe2b-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.491939 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.491622 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.494464 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.494430 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:11.494641 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.494600 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:11.494759 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.494688 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "kube-api-access-5t5wf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:11.495821 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.495803 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token" pod "75e01f27-9df6-4157-b1d8-77a71ae0f62d" (UID: "75e01f27-9df6-4157-b1d8-77a71ae0f62d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:11.547769 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.547732 5030 generic.go:356] "Generic (PLEG): container finished" podID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerID="0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92" exitCode=0 Oct 29 23:01:11.547985 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.547964 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" event={"ID":"75e01f27-9df6-4157-b1d8-77a71ae0f62d","Type":"ContainerDied","Data":"0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92"} Oct 29 23:01:11.548039 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.548007 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" event={"ID":"75e01f27-9df6-4157-b1d8-77a71ae0f62d","Type":"ContainerDied","Data":"a2b86bc7e107f1d10fc705f61aa995ad175f5c6dfa1893c10228219a6c23a855"} Oct 29 23:01:11.548039 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.548030 5030 scope.go:122] "RemoveContainer" containerID="0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92" Oct 29 23:01:11.548150 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.548139 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn" Oct 29 23:01:11.551906 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.551773 5030 generic.go:356] "Generic (PLEG): container finished" podID="f3423324-ad21-4998-a379-6baf63bd8f00" containerID="7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965" exitCode=0 Oct 29 23:01:11.552154 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.552062 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" event={"ID":"f3423324-ad21-4998-a379-6baf63bd8f00","Type":"ContainerDied","Data":"7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965"} Oct 29 23:01:11.554719 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.552275 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" event={"ID":"f3423324-ad21-4998-a379-6baf63bd8f00","Type":"ContainerDied","Data":"0bca72ae935f3287902885ce6598e426bd457a2d4c7a4de3ff149356b5cba91f"} Oct 29 23:01:11.554719 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.552372 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch" Oct 29 23:01:11.559387 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.559366 5030 generic.go:356] "Generic (PLEG): container finished" podID="191a1836-d335-4756-aa1e-fbc676effe2b" containerID="d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16" exitCode=0 Oct 29 23:01:11.559498 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.559410 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" event={"ID":"191a1836-d335-4756-aa1e-fbc676effe2b","Type":"ContainerDied","Data":"d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16"} Oct 29 23:01:11.559498 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.559429 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" Oct 29 23:01:11.559498 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.559447 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq" event={"ID":"191a1836-d335-4756-aa1e-fbc676effe2b","Type":"ContainerDied","Data":"fb0d07c47ba2364bea0b5639f748e324aee68d6d3c42dd548954af0e03da93b1"} Oct 29 23:01:11.569485 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.569461 5030 scope.go:122] "RemoveContainer" containerID="0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92" Oct 29 23:01:11.573959 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:11.573930 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92\": container with ID starting with 0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92 not found: ID does not exist" containerID="0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92" Oct 29 23:01:11.574157 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.574130 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92"} err="failed to get container status \"0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92\": rpc error: code = NotFound desc = could not find container \"0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92\": container with ID starting with 0d91362c67cfe38631dda758529e46f7330a762cd7e09c8ac5077a33230f4f92 not found: ID does not exist" Oct 29 23:01:11.574265 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.574254 5030 scope.go:122] "RemoveContainer" containerID="7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965" Oct 29 23:01:11.590671 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.590648 5030 scope.go:122] "RemoveContainer" containerID="7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965" Oct 29 23:01:11.591014 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:11.590992 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965\": container with ID starting with 7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965 not found: ID does not exist" containerID="7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965" Oct 29 23:01:11.591150 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.591019 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965"} err="failed to get container status \"7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965\": rpc error: code = NotFound desc = could not find container \"7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965\": container with ID starting with 7bf10b2b0734a3ec3c3514346a06132c3085ab4a1d7658804ffbdcf2b8515965 not found: ID does not exist" Oct 29 23:01:11.591150 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.591036 5030 scope.go:122] "RemoveContainer" containerID="d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16" Oct 29 23:01:11.593254 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.593232 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.593403 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.593258 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.593403 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.593280 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.593403 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.593293 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75e01f27-9df6-4157-b1d8-77a71ae0f62d-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.593403 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.593305 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t5wf\" (UniqueName: \"kubernetes.io/projected/75e01f27-9df6-4157-b1d8-77a71ae0f62d-kube-api-access-5t5wf\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:11.602133 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.602109 5030 scope.go:122] "RemoveContainer" containerID="d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16" Oct 29 23:01:11.602450 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:11.602427 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16\": container with ID starting with d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16 not found: ID does not exist" containerID="d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16" Oct 29 23:01:11.602531 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.602460 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16"} err="failed to get container status \"d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16\": rpc error: code = NotFound desc = could not find container \"d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16\": container with ID starting with d8bbf2afc7cb9f49dbfd979fff38cdad9f3e75549959b546e1f7bfd7949fef16 not found: ID does not exist" Oct 29 23:01:11.622426 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.622403 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn"] Oct 29 23:01:11.661229 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.661171 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-tz9hn"] Oct 29 23:01:11.811986 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.811956 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq"] Oct 29 23:01:11.828144 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.828118 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-nxjlq"] Oct 29 23:01:11.875550 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.875523 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch"] Oct 29 23:01:11.902023 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:11.902000 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-81f92d-openshift-default-9cbc48765-g5xch"] Oct 29 23:01:13.046562 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.046522 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" path="/var/lib/kubelet/pods/191a1836-d335-4756-aa1e-fbc676effe2b/volumes" Oct 29 23:01:13.047084 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.047024 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" path="/var/lib/kubelet/pods/75e01f27-9df6-4157-b1d8-77a71ae0f62d/volumes" Oct 29 23:01:13.047468 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.047451 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" path="/var/lib/kubelet/pods/f3423324-ad21-4998-a379-6baf63bd8f00/volumes" Oct 29 23:01:13.398550 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.398461 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc"] Oct 29 23:01:13.398735 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.398705 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" podUID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" containerName="maas-api" containerID="cri-o://072c0048248906662b0d96702d9795032562fd8c5a369841f6a8c74c84b73f1c" gracePeriod=30 Oct 29 23:01:13.493583 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.493546 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" podUID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.75:8443/health\": dial tcp 10.132.0.75:8443: connect: connection refused" Oct 29 23:01:13.580888 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.580819 5030 generic.go:356] "Generic (PLEG): container finished" podID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" containerID="072c0048248906662b0d96702d9795032562fd8c5a369841f6a8c74c84b73f1c" exitCode=0 Oct 29 23:01:13.581050 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.580934 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" event={"ID":"d944d8e4-7578-43f0-a586-c61f10d4f8f0","Type":"ContainerDied","Data":"072c0048248906662b0d96702d9795032562fd8c5a369841f6a8c74c84b73f1c"} Oct 29 23:01:13.789289 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.789248 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 23:01:13.823985 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.823952 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca\") pod \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " Oct 29 23:01:13.824170 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.823992 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs\") pod \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " Oct 29 23:01:13.824170 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.824036 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls\") pod \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " Oct 29 23:01:13.824170 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.824104 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv\" (UniqueName: \"kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv\") pod \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\" (UID: \"d944d8e4-7578-43f0-a586-c61f10d4f8f0\") " Oct 29 23:01:13.824414 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.824377 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca" pod "d944d8e4-7578-43f0-a586-c61f10d4f8f0" (UID: "d944d8e4-7578-43f0-a586-c61f10d4f8f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:13.824830 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.824803 5030 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d944d8e4-7578-43f0-a586-c61f10d4f8f0-openshift-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:13.826897 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.826867 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv" pod "d944d8e4-7578-43f0-a586-c61f10d4f8f0" (UID: "d944d8e4-7578-43f0-a586-c61f10d4f8f0"). InnerVolumeSpecName "kube-api-access-f8pfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:13.827112 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.827086 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs" pod "d944d8e4-7578-43f0-a586-c61f10d4f8f0" (UID: "d944d8e4-7578-43f0-a586-c61f10d4f8f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:01:13.827188 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.827124 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls" pod "d944d8e4-7578-43f0-a586-c61f10d4f8f0" (UID: "d944d8e4-7578-43f0-a586-c61f10d4f8f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:01:13.927498 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.927460 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:13.927498 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.927498 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d944d8e4-7578-43f0-a586-c61f10d4f8f0-maas-api-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:13.927940 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:13.927513 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8pfv\" (UniqueName: \"kubernetes.io/projected/d944d8e4-7578-43f0-a586-c61f10d4f8f0-kube-api-access-f8pfv\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:14.596048 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.595944 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" event={"ID":"d944d8e4-7578-43f0-a586-c61f10d4f8f0","Type":"ContainerDied","Data":"2515fed92eb9362431b431f4dafdcfe9ec989a9a459d2666681709209c8fd6eb"} Oct 29 23:01:14.596482 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.596225 5030 scope.go:122] "RemoveContainer" containerID="072c0048248906662b0d96702d9795032562fd8c5a369841f6a8c74c84b73f1c" Oct 29 23:01:14.596482 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.596432 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc" Oct 29 23:01:14.602322 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.601563 5030 generic.go:356] "Generic (PLEG): container finished" podID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerID="bdb6d1240d9f7cee759899fdd2888971052f3e2d06d2dbd3ce97a27da4b4f06a" exitCode=0 Oct 29 23:01:14.602322 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.601631 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" event={"ID":"a8b8b321-58ef-4dea-a685-7f17127a228a","Type":"ContainerDied","Data":"bdb6d1240d9f7cee759899fdd2888971052f3e2d06d2dbd3ce97a27da4b4f06a"} Oct 29 23:01:14.603811 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.603660 5030 generic.go:356] "Generic (PLEG): container finished" podID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerID="b8fac82d97f566103ca88767fd49eea451e255eb1c1a6b1d6ccdc59517cad63b" exitCode=0 Oct 29 23:01:14.603811 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.603774 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" event={"ID":"4e9f7211-013a-4d0d-8ffd-71a577250517","Type":"ContainerDied","Data":"b8fac82d97f566103ca88767fd49eea451e255eb1c1a6b1d6ccdc59517cad63b"} Oct 29 23:01:14.678072 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.677363 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc"] Oct 29 23:01:14.705443 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.704523 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-07eef2-6d96795696-w62nc"] Oct 29 23:01:14.827184 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.827155 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 23:01:14.853223 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.853191 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 23:01:14.858784 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.858760 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 23:01:14.975372 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975346 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.975569 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975385 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.975772 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975741 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.975893 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975772 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.975893 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975823 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.975893 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975870 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976068 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975901 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.976068 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975947 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976068 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.975983 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976068 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976024 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.976068 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976067 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976097 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.976309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976157 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976184 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976225 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976268 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976340 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.976556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976382 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976443 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976492 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.976556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976547 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976803 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976581 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976803 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976621 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j\") pod \"a8b8b321-58ef-4dea-a685-7f17127a228a\" (UID: \"a8b8b321-58ef-4dea-a685-7f17127a228a\") " Oct 29 23:01:14.976803 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976704 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.976803 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976734 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.977024 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976864 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.977024 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976906 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs\") pod \"7133acf0-63a8-4436-bda0-a9961775d342\" (UID: \"7133acf0-63a8-4436-bda0-a9961775d342\") " Oct 29 23:01:14.977024 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976916 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:14.977024 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.976953 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data\") pod \"4e9f7211-013a-4d0d-8ffd-71a577250517\" (UID: \"4e9f7211-013a-4d0d-8ffd-71a577250517\") " Oct 29 23:01:14.977221 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.977138 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.978025 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.977996 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.978025 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.978007 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:14.978205 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.978050 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:14.978205 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.978066 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e9f7211-013a-4d0d-8ffd-71a577250517-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:14.978312 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.978222 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.978755 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.978543 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:14.979000 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.978928 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.979100 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979056 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.979338 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979285 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:14.979464 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979349 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.980128 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979606 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.980128 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979729 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.980128 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979745 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.980128 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.979861 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.980128 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.980101 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.980128 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.980115 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:14.982658 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.982631 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "kube-api-access-2sdz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:14.982762 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.982690 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "kube-api-access-fcx8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:14.985790 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.985763 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.985790 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.985764 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.987041 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.987007 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:14.987533 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.987500 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:14.992039 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.990017 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:14.992039 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.990024 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:14.992039 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.992004 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo" pod "7133acf0-63a8-4436-bda0-a9961775d342" (UID: "7133acf0-63a8-4436-bda0-a9961775d342"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:14.992320 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.992296 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo" pod "a8b8b321-58ef-4dea-a685-7f17127a228a" (UID: "a8b8b321-58ef-4dea-a685-7f17127a228a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:14.992394 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:14.992378 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph" pod "4e9f7211-013a-4d0d-8ffd-71a577250517" (UID: "4e9f7211-013a-4d0d-8ffd-71a577250517"). InnerVolumeSpecName "kube-api-access-rlsph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:15.053507 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.053470 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" path="/var/lib/kubelet/pods/d944d8e4-7578-43f0-a586-c61f10d4f8f0/volumes" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079112 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079144 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079158 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079170 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079180 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079191 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079205 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlsph\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-kube-api-access-rlsph\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079218 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sdz4\" (UniqueName: \"kubernetes.io/projected/7133acf0-63a8-4436-bda0-a9961775d342-kube-api-access-2sdz4\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079226 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079255 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079263 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079271 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079284 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079297 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079305 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8b8b321-58ef-4dea-a685-7f17127a228a-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079314 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7133acf0-63a8-4436-bda0-a9961775d342-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079324 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079334 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079342 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7133acf0-63a8-4436-bda0-a9961775d342-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079349 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8b8b321-58ef-4dea-a685-7f17127a228a-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079360 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8b8b321-58ef-4dea-a685-7f17127a228a-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079372 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcx8j\" (UniqueName: \"kubernetes.io/projected/a8b8b321-58ef-4dea-a685-7f17127a228a-kube-api-access-fcx8j\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079381 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e9f7211-013a-4d0d-8ffd-71a577250517-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.079789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.079389 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7133acf0-63a8-4436-bda0-a9961775d342-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:15.624186 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.624149 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" Oct 29 23:01:15.624186 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.624169 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms" event={"ID":"4e9f7211-013a-4d0d-8ffd-71a577250517","Type":"ContainerDied","Data":"22aeffc2d847d18846d3bf6028765306cbc1c5303d5404cc8fb3219da63918f4"} Oct 29 23:01:15.624614 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.624260 5030 scope.go:122] "RemoveContainer" containerID="b8fac82d97f566103ca88767fd49eea451e255eb1c1a6b1d6ccdc59517cad63b" Oct 29 23:01:15.626968 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.626936 5030 generic.go:356] "Generic (PLEG): container finished" podID="7133acf0-63a8-4436-bda0-a9961775d342" containerID="dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3" exitCode=0 Oct 29 23:01:15.627086 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.626999 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" Oct 29 23:01:15.627086 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.627019 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" event={"ID":"7133acf0-63a8-4436-bda0-a9961775d342","Type":"ContainerDied","Data":"dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3"} Oct 29 23:01:15.627198 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.627098 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq" event={"ID":"7133acf0-63a8-4436-bda0-a9961775d342","Type":"ContainerDied","Data":"484c2e95a6d069c999cd6c146951889a2f7fe15de2f983de5cd631cfa7a54b37"} Oct 29 23:01:15.630774 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.630753 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" event={"ID":"a8b8b321-58ef-4dea-a685-7f17127a228a","Type":"ContainerDied","Data":"4c183acf25f1916ae0cd99181d572e59f09e39ce0765d5e051ceb2f7b80c8165"} Oct 29 23:01:15.632027 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.632002 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h" Oct 29 23:01:15.637740 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.637707 5030 scope.go:122] "RemoveContainer" containerID="dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3" Oct 29 23:01:15.648549 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.648530 5030 scope.go:122] "RemoveContainer" containerID="dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3" Oct 29 23:01:15.648931 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:15.648910 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3\": container with ID starting with dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3 not found: ID does not exist" containerID="dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3" Oct 29 23:01:15.649038 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.648936 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3"} err="failed to get container status \"dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3\": rpc error: code = NotFound desc = could not find container \"dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3\": container with ID starting with dd9bb26f52b422e0bb7c96a0dc71171df2b8b5df423cf055bb43696b9b2aacd3 not found: ID does not exist" Oct 29 23:01:15.649038 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.648952 5030 scope.go:122] "RemoveContainer" containerID="bdb6d1240d9f7cee759899fdd2888971052f3e2d06d2dbd3ce97a27da4b4f06a" Oct 29 23:01:15.722481 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.722452 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms"] Oct 29 23:01:15.756976 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.756948 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-qzcms"] Oct 29 23:01:15.845935 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.845902 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq"] Oct 29 23:01:15.851714 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.851690 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-xhvkq"] Oct 29 23:01:15.894005 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.893976 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h"] Oct 29 23:01:15.906925 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:15.906896 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a2e7c8-openshift-default-694c67bc49-24k5h"] Oct 29 23:01:16.119257 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:16.119172 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m"] Oct 29 23:01:16.119428 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:16.119381 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" containerID="cri-o://45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000" gracePeriod=30 Oct 29 23:01:16.146639 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:16.146610 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p"] Oct 29 23:01:16.146856 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:16.146815 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" containerID="cri-o://00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792" gracePeriod=30 Oct 29 23:01:17.047363 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:17.047329 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" path="/var/lib/kubelet/pods/4e9f7211-013a-4d0d-8ffd-71a577250517/volumes" Oct 29 23:01:17.047898 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:17.047878 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7133acf0-63a8-4436-bda0-a9961775d342" path="/var/lib/kubelet/pods/7133acf0-63a8-4436-bda0-a9961775d342/volumes" Oct 29 23:01:17.048385 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:17.048368 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" path="/var/lib/kubelet/pods/a8b8b321-58ef-4dea-a685-7f17127a228a/volumes" Oct 29 23:01:18.495294 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:18.495263 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:01:18.495765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:18.495314 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:01:19.121174 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:19.121141 5030 patch_prober.go:29] interesting pod/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:01:19.121394 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:19.121195 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:01:20.253639 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:20.253607 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:01:21.432344 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.432316 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 23:01:21.471790 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.471769 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 23:01:21.568688 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.568661 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.568948 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.568693 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.568948 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.568729 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.568948 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.568748 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.568948 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.568795 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.568948 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.568945 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.569222 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569008 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.569222 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569025 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.569222 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569094 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.569222 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569128 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.569222 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569156 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.569463 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569226 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.569463 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569321 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:21.569463 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569387 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.569463 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569425 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.569463 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569439 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.569463 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569460 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.569771 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569491 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.569771 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569551 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data\") pod \"202a1cb6-c685-4178-b908-6a53a7818269\" (UID: \"202a1cb6-c685-4178-b908-6a53a7818269\") " Oct 29 23:01:21.569771 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569620 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.569771 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569747 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.570017 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569772 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket\") pod \"ea863520-7d1a-4d24-b491-e1b9cc180044\" (UID: \"ea863520-7d1a-4d24-b491-e1b9cc180044\") " Oct 29 23:01:21.570017 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.569992 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.570321 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570188 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.570321 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570268 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.570592 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570314 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.570592 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570411 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.570592 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570435 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:21.570592 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570585 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570606 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202a1cb6-c685-4178-b908-6a53a7818269-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570620 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570634 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea863520-7d1a-4d24-b491-e1b9cc180044-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570647 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570660 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570674 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570688 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570702 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.570746 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.570714 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.572320 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.572178 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.572665 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.572637 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:21.572779 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.572733 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:21.572779 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.572763 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "kube-api-access-6ksx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:21.573117 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.573076 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "kube-api-access-lttmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:21.573244 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.573174 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:21.573244 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.573203 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token" pod "202a1cb6-c685-4178-b908-6a53a7818269" (UID: "202a1cb6-c685-4178-b908-6a53a7818269"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:21.573740 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.573722 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy" pod "ea863520-7d1a-4d24-b491-e1b9cc180044" (UID: "ea863520-7d1a-4d24-b491-e1b9cc180044"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672409 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672437 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672446 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202a1cb6-c685-4178-b908-6a53a7818269-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672455 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ksx5\" (UniqueName: \"kubernetes.io/projected/202a1cb6-c685-4178-b908-6a53a7818269-kube-api-access-6ksx5\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672464 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672472 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea863520-7d1a-4d24-b491-e1b9cc180044-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.672483 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672482 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lttmv\" (UniqueName: \"kubernetes.io/projected/ea863520-7d1a-4d24-b491-e1b9cc180044-kube-api-access-lttmv\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.673231 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.672490 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202a1cb6-c685-4178-b908-6a53a7818269-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:21.678590 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.678565 5030 generic.go:356] "Generic (PLEG): container finished" podID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerID="00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792" exitCode=0 Oct 29 23:01:21.678717 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.678616 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" Oct 29 23:01:21.678717 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.678651 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" event={"ID":"ea863520-7d1a-4d24-b491-e1b9cc180044","Type":"ContainerDied","Data":"00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792"} Oct 29 23:01:21.678717 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.678705 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p" event={"ID":"ea863520-7d1a-4d24-b491-e1b9cc180044","Type":"ContainerDied","Data":"439d2319c185ae12fb9a0200c33c6f3e27f1fb57baa1e3b3088e8c97fa14c2bd"} Oct 29 23:01:21.678830 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.678735 5030 scope.go:122] "RemoveContainer" containerID="00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792" Oct 29 23:01:21.680340 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.680314 5030 generic.go:356] "Generic (PLEG): container finished" podID="202a1cb6-c685-4178-b908-6a53a7818269" containerID="45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000" exitCode=0 Oct 29 23:01:21.680450 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.680347 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" Oct 29 23:01:21.680450 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.680362 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" event={"ID":"202a1cb6-c685-4178-b908-6a53a7818269","Type":"ContainerDied","Data":"45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000"} Oct 29 23:01:21.680450 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.680405 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m" event={"ID":"202a1cb6-c685-4178-b908-6a53a7818269","Type":"ContainerDied","Data":"e1b531f664a778b1de23eb25e5c6982fcd0c71391d8d726a7e93b22d647566ac"} Oct 29 23:01:21.691010 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.690948 5030 scope.go:122] "RemoveContainer" containerID="00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792" Oct 29 23:01:21.691256 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:21.691234 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792\": container with ID starting with 00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792 not found: ID does not exist" containerID="00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792" Oct 29 23:01:21.691316 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.691268 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792"} err="failed to get container status \"00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792\": rpc error: code = NotFound desc = could not find container \"00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792\": container with ID starting with 00b1f84793d8dc0846b09b9f97ed01163b15bd6ea023df8202bd2fc7c12a8792 not found: ID does not exist" Oct 29 23:01:21.691316 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.691290 5030 scope.go:122] "RemoveContainer" containerID="45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000" Oct 29 23:01:21.701785 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.701768 5030 scope.go:122] "RemoveContainer" containerID="45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000" Oct 29 23:01:21.702060 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:21.702042 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000\": container with ID starting with 45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000 not found: ID does not exist" containerID="45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000" Oct 29 23:01:21.702113 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.702067 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000"} err="failed to get container status \"45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000\": rpc error: code = NotFound desc = could not find container \"45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000\": container with ID starting with 45fd6e9605a3a86c15412f4f5cc0e235442c79192e988c677f006147ca8e1000 not found: ID does not exist" Oct 29 23:01:21.740020 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.739997 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p"] Oct 29 23:01:21.756635 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.756613 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-tws7p"] Oct 29 23:01:21.794753 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.794731 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m"] Oct 29 23:01:21.813261 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:21.813237 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-20f88f-openshift-default-68c49c98f9-6t89m"] Oct 29 23:01:23.046101 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:23.046072 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="202a1cb6-c685-4178-b908-6a53a7818269" path="/var/lib/kubelet/pods/202a1cb6-c685-4178-b908-6a53a7818269/volumes" Oct 29 23:01:23.046556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:23.046542 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" path="/var/lib/kubelet/pods/ea863520-7d1a-4d24-b491-e1b9cc180044/volumes" Oct 29 23:01:24.935892 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:24.935831 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc"] Oct 29 23:01:24.936291 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:24.936108 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" containerID="cri-o://531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed" gracePeriod=30 Oct 29 23:01:24.983156 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:24.983118 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs"] Oct 29 23:01:24.983474 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:24.983439 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" containerID="cri-o://1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969" gracePeriod=30 Oct 29 23:01:24.984304 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:24.984245 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs"] Oct 29 23:01:24.985410 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:24.984918 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" containerID="cri-o://ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c" gracePeriod=30 Oct 29 23:01:30.330547 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.330473 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 23:01:30.365009 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.364985 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 23:01:30.382200 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.382120 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.402494 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403289 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403321 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403362 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403406 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403434 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403458 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403493 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403521 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403569 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403594 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403625 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.403732 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403652 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405369 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405340 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.405454 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405389 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.405454 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405446 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405531 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405520 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405563 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405553 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405602 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405588 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.405696 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405682 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.405733 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405719 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405766 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405750 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405796 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405782 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket\") pod \"0a3a662f-78c1-486f-b827-fe223058403d\" (UID: \"0a3a662f-78c1-486f-b827-fe223058403d\") " Oct 29 23:01:30.405873 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405857 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.405926 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405913 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.405973 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.405959 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs\") pod \"8abd35f7-2058-48c1-bdff-25ed40018ef8\" (UID: \"8abd35f7-2058-48c1-bdff-25ed40018ef8\") " Oct 29 23:01:30.406066 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.406053 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo\") pod \"dec73450-2334-40a8-a461-20fa18c57542\" (UID: \"dec73450-2334-40a8-a461-20fa18c57542\") " Oct 29 23:01:30.408394 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.403217 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.408394 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.407787 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:30.409324 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.409294 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:30.409814 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.409782 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:30.410343 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410312 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:30.410617 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410596 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.410703 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410643 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.410703 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410657 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.410821 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410707 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "kube-api-access-dt79c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:30.411247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410903 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.411247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.410943 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.411247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.411106 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.411247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.411143 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.411247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.411228 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.411247 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.411237 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.411564 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.411295 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:01:30.411564 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.411439 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.412177 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.412151 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.412551 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.412525 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "kube-api-access-m7kp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:30.415577 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.415548 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.417093 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.417074 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.417198 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.417107 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:30.417198 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.417133 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token" pod "8abd35f7-2058-48c1-bdff-25ed40018ef8" (UID: "8abd35f7-2058-48c1-bdff-25ed40018ef8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:30.417424 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.417392 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy" pod "dec73450-2334-40a8-a461-20fa18c57542" (UID: "dec73450-2334-40a8-a461-20fa18c57542"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:01:30.417532 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.417464 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:01:30.417963 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.417939 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "kube-api-access-n8gvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:30.418636 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.418616 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token" pod "0a3a662f-78c1-486f-b827-fe223058403d" (UID: "0a3a662f-78c1-486f-b827-fe223058403d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:01:30.508065 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508030 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abd35f7-2058-48c1-bdff-25ed40018ef8-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508065 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508064 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508065 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508078 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508091 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508105 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508120 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7kp7\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-kube-api-access-m7kp7\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508133 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508147 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dec73450-2334-40a8-a461-20fa18c57542-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508159 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508171 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt79c\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-kube-api-access-dt79c\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508183 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a3a662f-78c1-486f-b827-fe223058403d-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508198 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508210 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508222 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508234 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508246 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508258 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a3a662f-78c1-486f-b827-fe223058403d-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508271 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508284 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8gvn\" (UniqueName: \"kubernetes.io/projected/0a3a662f-78c1-486f-b827-fe223058403d-kube-api-access-n8gvn\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508297 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508309 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dec73450-2334-40a8-a461-20fa18c57542-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508321 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508334 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abd35f7-2058-48c1-bdff-25ed40018ef8-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508347 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508359 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dec73450-2334-40a8-a461-20fa18c57542-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508374 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dec73450-2334-40a8-a461-20fa18c57542-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.508378 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.508387 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a3a662f-78c1-486f-b827-fe223058403d-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:01:30.753141 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.753106 5030 generic.go:356] "Generic (PLEG): container finished" podID="dec73450-2334-40a8-a461-20fa18c57542" containerID="ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c" exitCode=0 Oct 29 23:01:30.753346 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.753160 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" Oct 29 23:01:30.753346 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.753198 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" event={"ID":"dec73450-2334-40a8-a461-20fa18c57542","Type":"ContainerDied","Data":"ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c"} Oct 29 23:01:30.753346 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.753337 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs" event={"ID":"dec73450-2334-40a8-a461-20fa18c57542","Type":"ContainerDied","Data":"f15cf1cd048a7d41a614f7cb42b8b9bb505115957380b1274bc63c3a879686db"} Oct 29 23:01:30.753506 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.753370 5030 scope.go:122] "RemoveContainer" containerID="ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c" Oct 29 23:01:30.754900 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.754872 5030 generic.go:356] "Generic (PLEG): container finished" podID="0a3a662f-78c1-486f-b827-fe223058403d" containerID="1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969" exitCode=0 Oct 29 23:01:30.755042 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.754911 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" event={"ID":"0a3a662f-78c1-486f-b827-fe223058403d","Type":"ContainerDied","Data":"1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969"} Oct 29 23:01:30.755042 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.754938 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" Oct 29 23:01:30.755042 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.754954 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs" event={"ID":"0a3a662f-78c1-486f-b827-fe223058403d","Type":"ContainerDied","Data":"6b2f5c57cbe020bc414bc8970bed2f807fdc48bd210318c5970cf9e9905b37f7"} Oct 29 23:01:30.757290 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.757250 5030 generic.go:356] "Generic (PLEG): container finished" podID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerID="531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed" exitCode=0 Oct 29 23:01:30.757406 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.757319 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" Oct 29 23:01:30.757406 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.757369 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" event={"ID":"8abd35f7-2058-48c1-bdff-25ed40018ef8","Type":"ContainerDied","Data":"531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed"} Oct 29 23:01:30.757522 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.757407 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc" event={"ID":"8abd35f7-2058-48c1-bdff-25ed40018ef8","Type":"ContainerDied","Data":"0c4daf2a0559913ed4b07632d47f4b5c6e7305b99d2a728174fe1e5fb282d070"} Oct 29 23:01:30.768216 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.768089 5030 scope.go:122] "RemoveContainer" containerID="ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c" Oct 29 23:01:30.768519 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:30.768499 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c\": container with ID starting with ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c not found: ID does not exist" containerID="ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c" Oct 29 23:01:30.768598 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.768533 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c"} err="failed to get container status \"ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c\": rpc error: code = NotFound desc = could not find container \"ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c\": container with ID starting with ef82ddc39d7c28e790fc98817847e89faff1fca6ada20a488b4afdb1101b478c not found: ID does not exist" Oct 29 23:01:30.768598 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.768570 5030 scope.go:122] "RemoveContainer" containerID="1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969" Oct 29 23:01:30.780983 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.780957 5030 scope.go:122] "RemoveContainer" containerID="1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969" Oct 29 23:01:30.781308 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:30.781290 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969\": container with ID starting with 1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969 not found: ID does not exist" containerID="1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969" Oct 29 23:01:30.781374 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.781315 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969"} err="failed to get container status \"1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969\": rpc error: code = NotFound desc = could not find container \"1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969\": container with ID starting with 1596e7ec86ab839fa8c4bcf8542010725f99633deb59c1129e5e8c820ae34969 not found: ID does not exist" Oct 29 23:01:30.781374 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.781330 5030 scope.go:122] "RemoveContainer" containerID="531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed" Oct 29 23:01:30.790486 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.790468 5030 scope.go:122] "RemoveContainer" containerID="531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed" Oct 29 23:01:30.790788 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:30.790764 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed\": container with ID starting with 531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed not found: ID does not exist" containerID="531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed" Oct 29 23:01:30.790877 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.790790 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed"} err="failed to get container status \"531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed\": rpc error: code = NotFound desc = could not find container \"531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed\": container with ID starting with 531fefe1a0033e07ce4b306da866dafa31b1b8f9fd8c1dcf058d4604589e02ed not found: ID does not exist" Oct 29 23:01:30.839145 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.839120 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs"] Oct 29 23:01:30.851701 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.851676 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-vpnbs"] Oct 29 23:01:30.878742 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.878714 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc"] Oct 29 23:01:30.885240 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.885217 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-2wgvc"] Oct 29 23:01:30.920539 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.920511 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs"] Oct 29 23:01:30.923978 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:30.923950 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-07eef2-openshift-default-869b7995fc-trghs"] Oct 29 23:01:31.046041 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:31.046012 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a3a662f-78c1-486f-b827-fe223058403d" path="/var/lib/kubelet/pods/0a3a662f-78c1-486f-b827-fe223058403d/volumes" Oct 29 23:01:31.046497 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:31.046483 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" path="/var/lib/kubelet/pods/8abd35f7-2058-48c1-bdff-25ed40018ef8/volumes" Oct 29 23:01:31.046932 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:01:31.046918 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dec73450-2334-40a8-a461-20fa18c57542" path="/var/lib/kubelet/pods/dec73450-2334-40a8-a461-20fa18c57542/volumes" Oct 29 23:01:35.795175 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:01:35.795140 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:02:05.795293 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:02:05.795255 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:02:19.123817 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:19.123782 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz"] Oct 29 23:02:19.136080 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:19.136054 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9"] Oct 29 23:02:34.146546 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.146473 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" containerID="cri-o://818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0" gracePeriod=45 Oct 29 23:02:34.158345 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.158313 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" containerID="cri-o://81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126" gracePeriod=45 Oct 29 23:02:34.426499 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.426477 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 23:02:34.454855 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.454823 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 23:02:34.491088 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491060 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home\") pod \"cf68acac-097f-49e9-b78c-57b2b90b64d1\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " Oct 29 23:02:34.491253 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491117 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache\") pod \"cf68acac-097f-49e9-b78c-57b2b90b64d1\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " Oct 29 23:02:34.491253 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491149 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home\") pod \"240b0a83-97d1-466a-a216-3ccbb0180f5c\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " Oct 29 23:02:34.491253 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491217 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm\") pod \"240b0a83-97d1-466a-a216-3ccbb0180f5c\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " Oct 29 23:02:34.491417 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491265 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs\" (UniqueName: \"kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs\") pod \"240b0a83-97d1-466a-a216-3ccbb0180f5c\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " Oct 29 23:02:34.491417 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491292 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm\") pod \"cf68acac-097f-49e9-b78c-57b2b90b64d1\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " Oct 29 23:02:34.491417 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491322 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir\") pod \"cf68acac-097f-49e9-b78c-57b2b90b64d1\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " Oct 29 23:02:34.491417 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491338 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home" pod "cf68acac-097f-49e9-b78c-57b2b90b64d1" (UID: "cf68acac-097f-49e9-b78c-57b2b90b64d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.491624 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491444 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache" pod "cf68acac-097f-49e9-b78c-57b2b90b64d1" (UID: "cf68acac-097f-49e9-b78c-57b2b90b64d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.491624 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491452 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home" pod "240b0a83-97d1-466a-a216-3ccbb0180f5c" (UID: "240b0a83-97d1-466a-a216-3ccbb0180f5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.491624 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491519 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache\") pod \"240b0a83-97d1-466a-a216-3ccbb0180f5c\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " Oct 29 23:02:34.491624 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491598 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw\" (UniqueName: \"kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw\") pod \"cf68acac-097f-49e9-b78c-57b2b90b64d1\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " Oct 29 23:02:34.491624 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491617 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir" pod "cf68acac-097f-49e9-b78c-57b2b90b64d1" (UID: "cf68acac-097f-49e9-b78c-57b2b90b64d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491631 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf\" (UniqueName: \"kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf\") pod \"240b0a83-97d1-466a-a216-3ccbb0180f5c\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491698 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir\") pod \"240b0a83-97d1-466a-a216-3ccbb0180f5c\" (UID: \"240b0a83-97d1-466a-a216-3ccbb0180f5c\") " Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491737 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs\" (UniqueName: \"kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs\") pod \"cf68acac-097f-49e9-b78c-57b2b90b64d1\" (UID: \"cf68acac-097f-49e9-b78c-57b2b90b64d1\") " Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491828 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache" pod "240b0a83-97d1-466a-a216-3ccbb0180f5c" (UID: "240b0a83-97d1-466a-a216-3ccbb0180f5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.491945 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir" pod "240b0a83-97d1-466a-a216-3ccbb0180f5c" (UID: "240b0a83-97d1-466a-a216-3ccbb0180f5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.492291 5030 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-tmp-dir\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.492314 5030 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-model-cache\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.492333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.492329 5030 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-tmp-dir\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.492966 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.492342 5030 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-home\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.492966 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.492356 5030 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-model-cache\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.492966 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.492368 5030 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-home\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.494666 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.494638 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs" pod "cf68acac-097f-49e9-b78c-57b2b90b64d1" (UID: "cf68acac-097f-49e9-b78c-57b2b90b64d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:02:34.495155 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.495119 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs" pod "240b0a83-97d1-466a-a216-3ccbb0180f5c" (UID: "240b0a83-97d1-466a-a216-3ccbb0180f5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:02:34.495362 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.495135 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm" pod "cf68acac-097f-49e9-b78c-57b2b90b64d1" (UID: "cf68acac-097f-49e9-b78c-57b2b90b64d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.495606 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.495582 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf" pod "240b0a83-97d1-466a-a216-3ccbb0180f5c" (UID: "240b0a83-97d1-466a-a216-3ccbb0180f5c"). InnerVolumeSpecName "kube-api-access-fntcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:34.496507 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.496486 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm" pod "240b0a83-97d1-466a-a216-3ccbb0180f5c" (UID: "240b0a83-97d1-466a-a216-3ccbb0180f5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:34.497127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.497105 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw" pod "cf68acac-097f-49e9-b78c-57b2b90b64d1" (UID: "cf68acac-097f-49e9-b78c-57b2b90b64d1"). InnerVolumeSpecName "kube-api-access-svtnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:34.593578 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.593549 5030 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/240b0a83-97d1-466a-a216-3ccbb0180f5c-dshm\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.593730 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.593592 5030 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/240b0a83-97d1-466a-a216-3ccbb0180f5c-tls-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.593730 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.593601 5030 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf68acac-097f-49e9-b78c-57b2b90b64d1-dshm\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.593730 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.593610 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svtnw\" (UniqueName: \"kubernetes.io/projected/cf68acac-097f-49e9-b78c-57b2b90b64d1-kube-api-access-svtnw\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.593730 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.593620 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fntcf\" (UniqueName: \"kubernetes.io/projected/240b0a83-97d1-466a-a216-3ccbb0180f5c-kube-api-access-fntcf\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:34.593730 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:34.593628 5030 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf68acac-097f-49e9-b78c-57b2b90b64d1-tls-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:35.170239 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.170211 5030 generic.go:356] "Generic (PLEG): container finished" podID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerID="818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0" exitCode=0 Oct 29 23:02:35.170599 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.170270 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" Oct 29 23:02:35.170599 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.170294 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" event={"ID":"cf68acac-097f-49e9-b78c-57b2b90b64d1","Type":"ContainerDied","Data":"818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0"} Oct 29 23:02:35.170599 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.170334 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz" event={"ID":"cf68acac-097f-49e9-b78c-57b2b90b64d1","Type":"ContainerDied","Data":"f97e2b988d5830566b709951c3cf4f76ed35c44c04ce0a50f57cd6cb28c3c1a8"} Oct 29 23:02:35.170599 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.170350 5030 scope.go:122] "RemoveContainer" containerID="818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0" Oct 29 23:02:35.172279 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.172235 5030 generic.go:356] "Generic (PLEG): container finished" podID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerID="81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126" exitCode=0 Oct 29 23:02:35.172279 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.172268 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" Oct 29 23:02:35.172689 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.172264 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" event={"ID":"240b0a83-97d1-466a-a216-3ccbb0180f5c","Type":"ContainerDied","Data":"81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126"} Oct 29 23:02:35.172689 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.172428 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" event={"ID":"240b0a83-97d1-466a-a216-3ccbb0180f5c","Type":"ContainerDied","Data":"ec362a9ce98b715401f6cccf7c3944ae81095986650ed0471dfe6f9f42661bf3"} Oct 29 23:02:35.180122 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.180105 5030 scope.go:122] "RemoveContainer" containerID="818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0" Oct 29 23:02:35.180397 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:02:35.180378 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0\": container with ID starting with 818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0 not found: ID does not exist" containerID="818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0" Oct 29 23:02:35.180443 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.180404 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0"} err="failed to get container status \"818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0\": rpc error: code = NotFound desc = could not find container \"818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0\": container with ID starting with 818e3af73138ab00ebb21116fb253f83d813d33e00c2cc22387e6a00dacf9ee0 not found: ID does not exist" Oct 29 23:02:35.180443 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.180419 5030 scope.go:122] "RemoveContainer" containerID="81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126" Oct 29 23:02:35.188747 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.188731 5030 scope.go:122] "RemoveContainer" containerID="81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126" Oct 29 23:02:35.189039 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:02:35.189018 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126\": container with ID starting with 81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126 not found: ID does not exist" containerID="81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126" Oct 29 23:02:35.189123 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.189045 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126"} err="failed to get container status \"81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126\": rpc error: code = NotFound desc = could not find container \"81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126\": container with ID starting with 81d8e0a0c3b051c8def2c3251db1a0ef6fa656a117e88174ce7481fc8d3df126 not found: ID does not exist" Oct 29 23:02:35.224402 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.224379 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz"] Oct 29 23:02:35.236697 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.236678 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/distinct-w0-ffba57-kserve-56698887b8-557nz"] Oct 29 23:02:35.272903 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.272878 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9"] Oct 29 23:02:35.276335 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.276312 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9"] Oct 29 23:02:35.430983 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:35.430904 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ffba57/premium-w0-ffba57-kserve-bdd764b6f-9xss9" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded" Oct 29 23:02:35.794809 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:02:35.794777 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:02:37.045015 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:37.044983 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" path="/var/lib/kubelet/pods/240b0a83-97d1-466a-a216-3ccbb0180f5c/volumes" Oct 29 23:02:37.045572 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:37.045555 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" path="/var/lib/kubelet/pods/cf68acac-097f-49e9-b78c-57b2b90b64d1/volumes" Oct 29 23:02:38.749682 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:38.749651 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:02:47.749441 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:47.749397 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:02:50.045487 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:50.045453 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9"] Oct 29 23:02:50.045999 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:50.045694 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" containerID="cri-o://2ae8ab1eb31dcccb4d7b4dac910e5cb8a7eebf1a112ef4f05390ae07a9765097" gracePeriod=30 Oct 29 23:02:50.057694 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:50.057664 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd"] Oct 29 23:02:50.057940 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:50.057910 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" containerID="cri-o://b23ba6deb4e89040189294d68c7659f88512e35e3f986eea5e4a08be87a5dec6" gracePeriod=30 Oct 29 23:02:50.071535 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:50.071505 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm"] Oct 29 23:02:50.071807 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:50.071762 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" containerID="cri-o://7a00f6411f027715eaf9b3aabf7e8c472fb318b1ba8785acc12c6ae64739f34d" gracePeriod=30 Oct 29 23:02:51.301186 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:51.301142 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:02:51.301582 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:51.301213 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:02:52.891114 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:52.891080 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-848f956548-92rkf"] Oct 29 23:02:52.891530 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:52.891322 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-848f956548-92rkf" podUID="fc7883fd-ad17-4803-9278-3f05564850bf" containerName="manager" containerID="cri-o://e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f" gracePeriod=30 Oct 29 23:02:53.179287 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.179258 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 23:02:53.277919 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.277891 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs\") pod \"fc7883fd-ad17-4803-9278-3f05564850bf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " Oct 29 23:02:53.278106 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.277981 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert\") pod \"fc7883fd-ad17-4803-9278-3f05564850bf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " Oct 29 23:02:53.278106 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.278084 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq\" (UniqueName: \"kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq\") pod \"fc7883fd-ad17-4803-9278-3f05564850bf\" (UID: \"fc7883fd-ad17-4803-9278-3f05564850bf\") " Oct 29 23:02:53.280973 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.280947 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq" pod "fc7883fd-ad17-4803-9278-3f05564850bf" (UID: "fc7883fd-ad17-4803-9278-3f05564850bf"). InnerVolumeSpecName "kube-api-access-gj6qq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:53.281415 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.281393 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs" pod "fc7883fd-ad17-4803-9278-3f05564850bf" (UID: "fc7883fd-ad17-4803-9278-3f05564850bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:02:53.281583 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.281538 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert" pod "fc7883fd-ad17-4803-9278-3f05564850bf" (UID: "fc7883fd-ad17-4803-9278-3f05564850bf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:02:53.315919 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.315877 5030 generic.go:356] "Generic (PLEG): container finished" podID="fc7883fd-ad17-4803-9278-3f05564850bf" containerID="e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f" exitCode=0 Oct 29 23:02:53.315919 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.315911 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-848f956548-92rkf" Oct 29 23:02:53.316147 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.315943 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-848f956548-92rkf" event={"ID":"fc7883fd-ad17-4803-9278-3f05564850bf","Type":"ContainerDied","Data":"e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f"} Oct 29 23:02:53.316147 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.316016 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-848f956548-92rkf" event={"ID":"fc7883fd-ad17-4803-9278-3f05564850bf","Type":"ContainerDied","Data":"98a868158d0a22a7a8de3cb8d24ec1bae6c424645de5fb6cfe23bca32a68a3da"} Oct 29 23:02:53.316147 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.316048 5030 scope.go:122] "RemoveContainer" containerID="e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f" Oct 29 23:02:53.331413 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.331387 5030 scope.go:122] "RemoveContainer" containerID="e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f" Oct 29 23:02:53.331760 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:02:53.331742 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f\": container with ID starting with e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f not found: ID does not exist" containerID="e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f" Oct 29 23:02:53.331858 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.331768 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f"} err="failed to get container status \"e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f\": rpc error: code = NotFound desc = could not find container \"e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f\": container with ID starting with e06b5f0e27c0bc7d5bcea6875ec36aac14f74483feedd27067cea7150606027f not found: ID does not exist" Oct 29 23:02:53.379012 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.378983 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-848f956548-92rkf"] Oct 29 23:02:53.379459 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.379438 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:53.379535 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.379463 5030 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc7883fd-ad17-4803-9278-3f05564850bf-webhook-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:53.379535 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.379476 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj6qq\" (UniqueName: \"kubernetes.io/projected/fc7883fd-ad17-4803-9278-3f05564850bf-kube-api-access-gj6qq\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:53.385390 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:53.385363 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-848f956548-92rkf"] Oct 29 23:02:54.957088 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:54.957056 5030 patch_prober.go:29] interesting pod/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:02:54.957490 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:54.957112 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:02:55.045683 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.045651 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc7883fd-ad17-4803-9278-3f05564850bf" path="/var/lib/kubelet/pods/fc7883fd-ad17-4803-9278-3f05564850bf/volumes" Oct 29 23:02:55.332493 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.332468 5030 generic.go:356] "Generic (PLEG): container finished" podID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerID="b23ba6deb4e89040189294d68c7659f88512e35e3f986eea5e4a08be87a5dec6" exitCode=0 Oct 29 23:02:55.332615 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.332541 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" event={"ID":"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71","Type":"ContainerDied","Data":"b23ba6deb4e89040189294d68c7659f88512e35e3f986eea5e4a08be87a5dec6"} Oct 29 23:02:55.334348 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.334323 5030 generic.go:356] "Generic (PLEG): container finished" podID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerID="7a00f6411f027715eaf9b3aabf7e8c472fb318b1ba8785acc12c6ae64739f34d" exitCode=0 Oct 29 23:02:55.334468 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.334356 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" event={"ID":"ca490569-fa3a-4182-b39b-810264cbd3e3","Type":"ContainerDied","Data":"7a00f6411f027715eaf9b3aabf7e8c472fb318b1ba8785acc12c6ae64739f34d"} Oct 29 23:02:55.336112 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.335995 5030 generic.go:356] "Generic (PLEG): container finished" podID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerID="2ae8ab1eb31dcccb4d7b4dac910e5cb8a7eebf1a112ef4f05390ae07a9765097" exitCode=0 Oct 29 23:02:55.336112 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.336066 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" event={"ID":"fca5ff35-abae-4a7a-8077-fdafd607699a","Type":"ContainerDied","Data":"2ae8ab1eb31dcccb4d7b4dac910e5cb8a7eebf1a112ef4f05390ae07a9765097"} Oct 29 23:02:55.374655 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.374623 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 23:02:55.440977 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.440953 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 23:02:55.443143 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.443121 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503047 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503089 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503124 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503163 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503195 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503227 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503263 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503329 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503364 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503401 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503456 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503492 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503544 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503582 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503617 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503645 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503699 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503743 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503783 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503813 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.503905 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503917 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket\") pod \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\" (UID: \"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71\") " Oct 29 23:02:55.505111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.503965 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.505111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.504006 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.505111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.504053 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.505111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.504086 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.505111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.504121 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy\") pod \"ca490569-fa3a-4182-b39b-810264cbd3e3\" (UID: \"ca490569-fa3a-4182-b39b-810264cbd3e3\") " Oct 29 23:02:55.505111 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.504148 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy\") pod \"fca5ff35-abae-4a7a-8077-fdafd607699a\" (UID: \"fca5ff35-abae-4a7a-8077-fdafd607699a\") " Oct 29 23:02:55.505427 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.505315 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.505680 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.505930 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.506077 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.506229 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.506616 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.507044 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.507901 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.510207 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.512399 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.512588 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.515956 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.516545 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.516820 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.516902 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.518148 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.518212 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.519203 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "kube-api-access-wm5j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.520626 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.532238 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.532263 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "kube-api-access-f9cjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.532356 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:02:55.550889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.534093 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo" pod "ca490569-fa3a-4182-b39b-810264cbd3e3" (UID: "ca490569-fa3a-4182-b39b-810264cbd3e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:02:55.552385 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.552354 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.552596 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.552429 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token" pod "fca5ff35-abae-4a7a-8077-fdafd607699a" (UID: "fca5ff35-abae-4a7a-8077-fdafd607699a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:55.553495 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.552603 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "kube-api-access-gg5hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:02:55.554729 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.554706 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy" pod "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" (UID: "2e4ec2cb-2934-4f04-95e0-dd06b21cfa71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:02:55.605530 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605496 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605530 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605528 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605530 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605540 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605552 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca490569-fa3a-4182-b39b-810264cbd3e3-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605563 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605575 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605587 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605600 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605612 5030 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fca5ff35-abae-4a7a-8077-fdafd607699a-istiod-ca-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605623 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605634 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605645 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605657 5030 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605668 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca490569-fa3a-4182-b39b-810264cbd3e3-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605682 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm5j9\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-kube-api-access-wm5j9\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605695 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605706 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605717 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605727 5030 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-data\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605739 5030 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-podinfo\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605752 5030 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-envoy\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605765 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605780 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg5hl\" (UniqueName: \"kubernetes.io/projected/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-kube-api-access-gg5hl\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605793 5030 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71-credential-socket\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605805 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fca5ff35-abae-4a7a-8077-fdafd607699a-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605818 5030 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fca5ff35-abae-4a7a-8077-fdafd607699a-workload-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:55.605890 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:55.605831 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9cjp\" (UniqueName: \"kubernetes.io/projected/ca490569-fa3a-4182-b39b-810264cbd3e3-kube-api-access-f9cjp\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:02:56.346810 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.346724 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" Oct 29 23:02:56.346810 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.346740 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9" event={"ID":"fca5ff35-abae-4a7a-8077-fdafd607699a","Type":"ContainerDied","Data":"ed041b71072135e412c8d710f46b014043cdd568dba38d9c9d64b8a0cb3ecf58"} Oct 29 23:02:56.347472 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.346812 5030 scope.go:122] "RemoveContainer" containerID="2ae8ab1eb31dcccb4d7b4dac910e5cb8a7eebf1a112ef4f05390ae07a9765097" Oct 29 23:02:56.349852 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.349812 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" Oct 29 23:02:56.349972 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.349831 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd" event={"ID":"2e4ec2cb-2934-4f04-95e0-dd06b21cfa71","Type":"ContainerDied","Data":"82b3feb5ced308e466b7521004e84fc484676e7d36fbd1ae93683ceefe84fc93"} Oct 29 23:02:56.360080 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.359684 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" event={"ID":"ca490569-fa3a-4182-b39b-810264cbd3e3","Type":"ContainerDied","Data":"16794d13128efac3dbac1ab575b902a0a9384c8437797f75655278d5f52bfcc4"} Oct 29 23:02:56.360080 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.359722 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm" Oct 29 23:02:56.365965 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.365898 5030 scope.go:122] "RemoveContainer" containerID="b23ba6deb4e89040189294d68c7659f88512e35e3f986eea5e4a08be87a5dec6" Oct 29 23:02:56.381859 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.381818 5030 scope.go:122] "RemoveContainer" containerID="7a00f6411f027715eaf9b3aabf7e8c472fb318b1ba8785acc12c6ae64739f34d" Oct 29 23:02:56.426833 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.426793 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9"] Oct 29 23:02:56.441269 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.441236 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-smfr9"] Oct 29 23:02:56.474864 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.463760 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd"] Oct 29 23:02:56.474864 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.469884 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-mppnd"] Oct 29 23:02:56.546199 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.546165 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm"] Oct 29 23:02:56.577965 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:56.571965 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ffba57-openshift-default-9778845f9-zg9nm"] Oct 29 23:02:57.058684 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:57.058648 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" path="/var/lib/kubelet/pods/2e4ec2cb-2934-4f04-95e0-dd06b21cfa71/volumes" Oct 29 23:02:57.060643 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:57.060617 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" path="/var/lib/kubelet/pods/ca490569-fa3a-4182-b39b-810264cbd3e3/volumes" Oct 29 23:02:57.069563 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:02:57.069540 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" path="/var/lib/kubelet/pods/fca5ff35-abae-4a7a-8077-fdafd607699a/volumes" Oct 29 23:03:05.793919 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:03:05.793876 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:03:24.862504 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:03:24.862477 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:03:24.901394 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:03:24.901369 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:03:35.794127 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:03:35.794089 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:04:01.424350 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:01.424306 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:04:04.358112 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:04.358017 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:04:04.358596 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:04.358122 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-8m9xv" podUID="b6326e02-98c1-4554-8590-11a71357b8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:04:05.794018 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:04:05.793982 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:04:18.732949 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:18.732914 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:04:23.107928 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:23.107894 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:04:35.794305 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:04:35.794270 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:04:38.743306 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:38.743272 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:04:54.300781 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:04:54.300743 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:05:05.793950 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:05:05.793914 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:05:17.813434 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:05:17.813396 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:05:28.121561 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:05:28.121518 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:05:32.634655 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:05:32.634620 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:05:35.794780 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:05:35.794704 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:05:49.732515 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:05:49.732466 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:05:54.595087 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:05:54.595036 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:06:05.794903 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:06:05.794865 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:06:11.232309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:11.232270 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:06:15.935611 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:15.935575 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:06:31.345068 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:31.345033 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:06:35.869578 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:06:35.869537 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:06:36.394862 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:36.394299 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:06:37.965224 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:37.965187 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4"] Oct 29 23:06:37.965826 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:37.965391 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" podUID="2176fdbe-2a1d-4b30-9959-cc00cc909c84" containerName="discovery" containerID="cri-o://188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec" gracePeriod=30 Oct 29 23:06:38.265562 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.265534 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 23:06:38.357074 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357046 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.357249 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357108 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.357249 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357184 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.357406 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357371 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.357502 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357481 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.357695 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357679 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.357900 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357884 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs\") pod \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\" (UID: \"2176fdbe-2a1d-4b30-9959-cc00cc909c84\") " Oct 29 23:06:38.358024 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.357725 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:06:38.358532 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.358509 5030 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-ca-configmap\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.360306 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.360275 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:06:38.360779 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.360748 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:06:38.360942 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.360780 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:06:38.360942 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.360872 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:06:38.361060 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.360950 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "kube-api-access-m5ftz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:06:38.361189 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.361174 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts" pod "2176fdbe-2a1d-4b30-9959-cc00cc909c84" (UID: "2176fdbe-2a1d-4b30-9959-cc00cc909c84"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:06:38.459310 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.459284 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5ftz\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-kube-api-access-m5ftz\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.459310 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.459308 5030 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-cacerts\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.459475 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.459317 5030 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-token\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.459475 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.459326 5030 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-csr-dns-cert\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.459475 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.459335 5030 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2176fdbe-2a1d-4b30-9959-cc00cc909c84-local-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.459475 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.459343 5030 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2176fdbe-2a1d-4b30-9959-cc00cc909c84-istio-kubeconfig\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:06:38.939889 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.939831 5030 generic.go:356] "Generic (PLEG): container finished" podID="2176fdbe-2a1d-4b30-9959-cc00cc909c84" containerID="188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec" exitCode=0 Oct 29 23:06:38.940182 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.939887 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" event={"ID":"2176fdbe-2a1d-4b30-9959-cc00cc909c84","Type":"ContainerDied","Data":"188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec"} Oct 29 23:06:38.940182 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.939915 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" Oct 29 23:06:38.940182 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.939970 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4" event={"ID":"2176fdbe-2a1d-4b30-9959-cc00cc909c84","Type":"ContainerDied","Data":"aa3eec4e3d75dfe6d11c92c6ebd3a77e5d9a83326c98823e8b3e5bec43bfa2ea"} Oct 29 23:06:38.940182 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.939997 5030 scope.go:122] "RemoveContainer" containerID="188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec" Oct 29 23:06:38.951003 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.950987 5030 scope.go:122] "RemoveContainer" containerID="188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec" Oct 29 23:06:38.951262 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:06:38.951241 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec\": container with ID starting with 188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec not found: ID does not exist" containerID="188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec" Oct 29 23:06:38.951328 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:38.951270 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec"} err="failed to get container status \"188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec\": rpc error: code = NotFound desc = could not find container \"188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec\": container with ID starting with 188d1ffdd611fbb14a5a1fb32d78bcb3e99384e723573e9520754b5d9fe6d8ec not found: ID does not exist" Oct 29 23:06:39.006296 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:39.006269 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4"] Oct 29 23:06:39.019309 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:39.019284 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dnq4"] Oct 29 23:06:39.045682 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:39.045659 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2176fdbe-2a1d-4b30-9959-cc00cc909c84" path="/var/lib/kubelet/pods/2176fdbe-2a1d-4b30-9959-cc00cc909c84/volumes" Oct 29 23:06:51.619347 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:06:51.619306 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:07:02.247221 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:07:02.247191 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:07:05.796334 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:07:05.796294 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:07:07.440754 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:07:07.440716 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:07:24.292316 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:07:24.292282 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:07:29.226528 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:07:29.226497 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:07:35.794611 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:07:35.794576 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:07:48.124688 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:07:48.124646 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:07:52.896565 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:07:52.896529 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:08:05.794787 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:08:05.794748 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:08:06.636720 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:08:06.636685 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:08:20.839260 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:08:20.839229 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:08:24.909348 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:08:24.909319 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:08:24.940624 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:08:24.940597 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:08:35.799600 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:08:35.799521 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:08:35.840956 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:08:35.840925 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:08:53.456987 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:08:53.456955 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:09:05.793399 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:09:05.793361 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:09:08.856823 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:08.856786 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:09:27.521566 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:27.521526 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:09:29.391986 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:29.391952 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:09:29.392609 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:29.392577 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:09:32.896354 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:32.896314 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:09:35.795194 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:09:35.795151 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:09:39.665930 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:39.665890 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np"] Oct 29 23:09:39.666348 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:39.666065 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" podUID="c2712f7b-5e48-4f61-86e7-c4090f15d8de" containerName="manager" containerID="cri-o://299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3" gracePeriod=10 Oct 29 23:09:39.932060 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:39.932004 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 23:09:40.043996 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.043968 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9\" (UniqueName: \"kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9\") pod \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " Oct 29 23:09:40.044167 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.044071 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume\") pod \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\" (UID: \"c2712f7b-5e48-4f61-86e7-c4090f15d8de\") " Oct 29 23:09:40.044669 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.044642 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume" pod "c2712f7b-5e48-4f61-86e7-c4090f15d8de" (UID: "c2712f7b-5e48-4f61-86e7-c4090f15d8de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:09:40.046194 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.046174 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9" pod "c2712f7b-5e48-4f61-86e7-c4090f15d8de" (UID: "c2712f7b-5e48-4f61-86e7-c4090f15d8de"). InnerVolumeSpecName "kube-api-access-vn2m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:09:40.076832 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.076797 5030 generic.go:356] "Generic (PLEG): container finished" podID="c2712f7b-5e48-4f61-86e7-c4090f15d8de" containerID="299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3" exitCode=0 Oct 29 23:09:40.076984 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.076879 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" Oct 29 23:09:40.076984 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.076895 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" event={"ID":"c2712f7b-5e48-4f61-86e7-c4090f15d8de","Type":"ContainerDied","Data":"299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3"} Oct 29 23:09:40.077107 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.076993 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np" event={"ID":"c2712f7b-5e48-4f61-86e7-c4090f15d8de","Type":"ContainerDied","Data":"4355f8f76d270e20b2fd33faec7f45ee29c9e235ca04c0933ecef3453b373ae8"} Oct 29 23:09:40.077107 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.077025 5030 scope.go:122] "RemoveContainer" containerID="299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3" Oct 29 23:09:40.089465 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.089445 5030 scope.go:122] "RemoveContainer" containerID="299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3" Oct 29 23:09:40.089727 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:09:40.089702 5030 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3\": container with ID starting with 299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3 not found: ID does not exist" containerID="299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3" Oct 29 23:09:40.089871 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.089731 5030 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3"} err="failed to get container status \"299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3\": rpc error: code = NotFound desc = could not find container \"299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3\": container with ID starting with 299846cdd70a53bf81a9ebd2f6a6c419378271ff609a963ab475a41113df17d3 not found: ID does not exist" Oct 29 23:09:40.129351 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.129314 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np"] Oct 29 23:09:40.134684 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.134660 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-5g8np"] Oct 29 23:09:40.145173 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.145151 5030 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2712f7b-5e48-4f61-86e7-c4090f15d8de-extensions-socket-volume\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:09:40.145173 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:40.145174 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn2m9\" (UniqueName: \"kubernetes.io/projected/c2712f7b-5e48-4f61-86e7-c4090f15d8de-kube-api-access-vn2m9\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:09:41.044684 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:09:41.044653 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2712f7b-5e48-4f61-86e7-c4090f15d8de" path="/var/lib/kubelet/pods/c2712f7b-5e48-4f61-86e7-c4090f15d8de/volumes" Oct 29 23:10:05.794751 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:10:05.794713 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:10:35.794566 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:10:35.794522 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:10:42.810517 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.810483 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc"] Oct 29 23:10:42.811027 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811012 5030 cpu_manager.go:414] "containerMap: removing container" podUID="51e10dc5-da26-495c-8e26-574a86c666b0" containerName="maas-api" Oct 29 23:10:42.811027 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811027 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51e10dc5-da26-495c-8e26-574a86c666b0" containerName="maas-api" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811040 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811044 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811056 5030 cpu_manager.go:414] "containerMap: removing container" podUID="265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" containerName="maas-api" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811061 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" containerName="maas-api" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811072 5030 cpu_manager.go:414] "containerMap: removing container" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811076 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811086 5030 cpu_manager.go:414] "containerMap: removing container" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811090 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811097 5030 cpu_manager.go:414] "containerMap: removing container" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811101 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811108 5030 cpu_manager.go:414] "containerMap: removing container" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811112 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" Oct 29 23:10:42.811116 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811120 5030 cpu_manager.go:414] "containerMap: removing container" podUID="c2712f7b-5e48-4f61-86e7-c4090f15d8de" containerName="manager" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811124 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2712f7b-5e48-4f61-86e7-c4090f15d8de" containerName="manager" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811131 5030 cpu_manager.go:414] "containerMap: removing container" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811134 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811142 5030 cpu_manager.go:414] "containerMap: removing container" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811146 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811155 5030 cpu_manager.go:414] "containerMap: removing container" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811160 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811166 5030 cpu_manager.go:414] "containerMap: removing container" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811170 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811177 5030 cpu_manager.go:414] "containerMap: removing container" podUID="2176fdbe-2a1d-4b30-9959-cc00cc909c84" containerName="discovery" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811181 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2176fdbe-2a1d-4b30-9959-cc00cc909c84" containerName="discovery" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811188 5030 cpu_manager.go:414] "containerMap: removing container" podUID="fc7883fd-ad17-4803-9278-3f05564850bf" containerName="manager" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811192 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc7883fd-ad17-4803-9278-3f05564850bf" containerName="manager" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811198 5030 cpu_manager.go:414] "containerMap: removing container" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811202 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811209 5030 cpu_manager.go:414] "containerMap: removing container" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811213 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811221 5030 cpu_manager.go:414] "containerMap: removing container" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811225 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811233 5030 cpu_manager.go:414] "containerMap: removing container" podUID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" containerName="maas-api" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811237 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" containerName="maas-api" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811244 5030 cpu_manager.go:414] "containerMap: removing container" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811248 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811261 5030 cpu_manager.go:414] "containerMap: removing container" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811265 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811272 5030 cpu_manager.go:414] "containerMap: removing container" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811276 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811283 5030 cpu_manager.go:414] "containerMap: removing container" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811287 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811294 5030 cpu_manager.go:414] "containerMap: removing container" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811298 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811311 5030 cpu_manager.go:414] "containerMap: removing container" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811316 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811341 5030 cpu_manager.go:414] "containerMap: removing container" podUID="193b43e0-1296-4296-b7af-001487290791" containerName="maas-api" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811346 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="193b43e0-1296-4296-b7af-001487290791" containerName="maas-api" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811355 5030 cpu_manager.go:414] "containerMap: removing container" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811360 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811367 5030 cpu_manager.go:414] "containerMap: removing container" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811372 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811379 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811383 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811457 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a3a662f-78c1-486f-b827-fe223058403d" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811467 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2712f7b-5e48-4f61-86e7-c4090f15d8de" containerName="manager" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811472 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="240b0a83-97d1-466a-a216-3ccbb0180f5c" containerName="main" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811478 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="344cdc2c-85cc-4eb5-bb00-523b63b26b2c" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811484 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8b8b321-58ef-4dea-a685-7f17127a228a" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811491 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="75e01f27-9df6-4157-b1d8-77a71ae0f62d" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811498 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="2176fdbe-2a1d-4b30-9959-cc00cc909c84" containerName="discovery" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811504 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc7883fd-ad17-4803-9278-3f05564850bf" containerName="manager" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811508 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="dec73450-2334-40a8-a461-20fa18c57542" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811513 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca490569-fa3a-4182-b39b-810264cbd3e3" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811519 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea863520-7d1a-4d24-b491-e1b9cc180044" containerName="istio-proxy" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811525 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="d944d8e4-7578-43f0-a586-c61f10d4f8f0" containerName="maas-api" Oct 29 23:10:42.811508 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811530 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="202a1cb6-c685-4178-b908-6a53a7818269" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811537 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811542 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e4ec2cb-2934-4f04-95e0-dd06b21cfa71" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811547 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3423324-ad21-4998-a379-6baf63bd8f00" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811552 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="8abd35f7-2058-48c1-bdff-25ed40018ef8" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811557 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e9f7211-013a-4d0d-8ffd-71a577250517" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811564 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="7133acf0-63a8-4436-bda0-a9961775d342" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811571 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="193b43e0-1296-4296-b7af-001487290791" containerName="maas-api" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811576 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf68acac-097f-49e9-b78c-57b2b90b64d1" containerName="main" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811582 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="fca5ff35-abae-4a7a-8077-fdafd607699a" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811587 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="51e10dc5-da26-495c-8e26-574a86c666b0" containerName="maas-api" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811592 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811597 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="265dfb5f-4ce3-4b9d-8bb4-acdd48bf1273" containerName="maas-api" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811601 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0661d7c-66fc-4a2b-8251-60f6b7bc0c82" containerName="main" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811607 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="191a1836-d335-4756-aa1e-fbc676effe2b" containerName="istio-proxy" Oct 29 23:10:42.813159 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.811611 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="947f7f47-a7ce-419e-8b65-3f8c74cb608f" containerName="main" Oct 29 23:10:42.814996 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.814979 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:42.843414 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.843388 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vwgmk\"" Oct 29 23:10:42.845955 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.845930 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc"] Oct 29 23:10:42.959503 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.959481 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lml74\" (UniqueName: \"kubernetes.io/projected/74ecea0c-0833-4168-8e4b-35dc438ad4de-kube-api-access-lml74\") pod \"kuadrant-operator-controller-manager-57bb9bfff-kwjnc\" (UID: \"74ecea0c-0833-4168-8e4b-35dc438ad4de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:42.959651 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:42.959618 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74ecea0c-0833-4168-8e4b-35dc438ad4de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-kwjnc\" (UID: \"74ecea0c-0833-4168-8e4b-35dc438ad4de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:43.060691 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:43.060625 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lml74\" (UniqueName: \"kubernetes.io/projected/74ecea0c-0833-4168-8e4b-35dc438ad4de-kube-api-access-lml74\") pod \"kuadrant-operator-controller-manager-57bb9bfff-kwjnc\" (UID: \"74ecea0c-0833-4168-8e4b-35dc438ad4de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:43.060817 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:43.060707 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74ecea0c-0833-4168-8e4b-35dc438ad4de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-kwjnc\" (UID: \"74ecea0c-0833-4168-8e4b-35dc438ad4de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:43.061051 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:43.061036 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74ecea0c-0833-4168-8e4b-35dc438ad4de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-kwjnc\" (UID: \"74ecea0c-0833-4168-8e4b-35dc438ad4de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:43.084410 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:43.084381 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lml74\" (UniqueName: \"kubernetes.io/projected/74ecea0c-0833-4168-8e4b-35dc438ad4de-kube-api-access-lml74\") pod \"kuadrant-operator-controller-manager-57bb9bfff-kwjnc\" (UID: \"74ecea0c-0833-4168-8e4b-35dc438ad4de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:43.127783 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:43.127747 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:43.460220 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:43.460197 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc"] Oct 29 23:10:43.462528 ip-10-0-136-228 kubenswrapper[5030]: W1029 23:10:43.462502 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ecea0c_0833_4168_8e4b_35dc438ad4de.slice/crio-cee81e0c11a3208bb7147b5e866339664cafdcce04748e2381f9c9cfcb8f425b WatchSource:0}: Error finding container cee81e0c11a3208bb7147b5e866339664cafdcce04748e2381f9c9cfcb8f425b: Status 404 returned error can't find the container with id cee81e0c11a3208bb7147b5e866339664cafdcce04748e2381f9c9cfcb8f425b Oct 29 23:10:44.434336 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:44.434302 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" event={"ID":"74ecea0c-0833-4168-8e4b-35dc438ad4de","Type":"ContainerStarted","Data":"967682992c144b429d57195d3685abd21c0548cee5dce650469a422c78e0f2ca"} Oct 29 23:10:44.434735 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:44.434349 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" event={"ID":"74ecea0c-0833-4168-8e4b-35dc438ad4de","Type":"ContainerStarted","Data":"cee81e0c11a3208bb7147b5e866339664cafdcce04748e2381f9c9cfcb8f425b"} Oct 29 23:10:44.434735 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:44.434403 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:10:44.504286 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:44.504242 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" podStartSLOduration=2.504230337 podStartE2EDuration="2.504230337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:10:44.491439915 +0000 UTC m=+2241.604085503" watchObservedRunningTime="2026-10-29 23:10:44.504230337 +0000 UTC m=+2241.616875925" Oct 29 23:10:55.444334 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:10:55.444304 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-kwjnc" Oct 29 23:11:01.202322 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:11:01.202290 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:11:05.793972 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:11:05.793936 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:11:16.624952 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:11:16.624918 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:11:35.794318 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:11:35.794239 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:11:41.695858 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:11:41.695810 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:11:56.469897 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:11:56.469855 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:12:05.794726 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:12:05.794682 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:12:10.723063 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:12:10.723035 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:12:24.231205 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:12:24.231173 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:12:35.796224 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:12:35.796181 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:12:37.556565 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:12:37.556517 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:12:49.528554 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:12:49.528522 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:13:05.794808 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:13:05.794763 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:13:06.709127 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:06.709085 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:13:22.109155 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:22.109124 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:13:25.187232 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:25.187202 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:13:25.194650 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:25.194627 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:13:35.329263 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:35.329230 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:13:35.806442 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:13:35.806405 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:13:42.195137 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:42.195095 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:13:52.113088 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:13:52.113053 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:14:05.528418 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:05.528381 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:14:05.798713 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:14:05.798611 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:14:22.502264 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:22.502223 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:14:32.606443 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:32.606411 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:14:35.794929 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:14:35.794821 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:14:38.882379 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:38.882343 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4"] Oct 29 23:14:38.882899 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:38.882556 5030 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" podUID="2802b231-5ab0-43cd-85f1-c6c4c68c331a" containerName="maas-api" containerID="cri-o://2865294ecd2a0ba5d564c56f3dfab4fb520f074021ba1e3a30c5fdfb71460d4e" gracePeriod=30 Oct 29 23:14:40.046062 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.046027 5030 generic.go:356] "Generic (PLEG): container finished" podID="2802b231-5ab0-43cd-85f1-c6c4c68c331a" containerID="2865294ecd2a0ba5d564c56f3dfab4fb520f074021ba1e3a30c5fdfb71460d4e" exitCode=0 Oct 29 23:14:40.046421 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.046065 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" event={"ID":"2802b231-5ab0-43cd-85f1-c6c4c68c331a","Type":"ContainerDied","Data":"2865294ecd2a0ba5d564c56f3dfab4fb520f074021ba1e3a30c5fdfb71460d4e"} Oct 29 23:14:40.144584 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.144559 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 23:14:40.282140 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.282109 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca\") pod \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " Oct 29 23:14:40.282356 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.282164 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs\") pod \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " Oct 29 23:14:40.282356 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.282297 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p\" (UniqueName: \"kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p\") pod \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " Oct 29 23:14:40.282465 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.282369 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls\") pod \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\" (UID: \"2802b231-5ab0-43cd-85f1-c6c4c68c331a\") " Oct 29 23:14:40.282567 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.282542 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca" pod "2802b231-5ab0-43cd-85f1-c6c4c68c331a" (UID: "2802b231-5ab0-43cd-85f1-c6c4c68c331a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:14:40.282891 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.282872 5030 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2802b231-5ab0-43cd-85f1-c6c4c68c331a-openshift-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:40.285218 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.285189 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls" pod "2802b231-5ab0-43cd-85f1-c6c4c68c331a" (UID: "2802b231-5ab0-43cd-85f1-c6c4c68c331a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:14:40.285328 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.285307 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs" pod "2802b231-5ab0-43cd-85f1-c6c4c68c331a" (UID: "2802b231-5ab0-43cd-85f1-c6c4c68c331a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:14:40.285441 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.285425 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p" pod "2802b231-5ab0-43cd-85f1-c6c4c68c331a" (UID: "2802b231-5ab0-43cd-85f1-c6c4c68c331a"). InnerVolumeSpecName "kube-api-access-qv74p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:14:40.383920 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.383892 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv74p\" (UniqueName: \"kubernetes.io/projected/2802b231-5ab0-43cd-85f1-c6c4c68c331a-kube-api-access-qv74p\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:40.383920 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.383915 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-maas-api-tls\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:40.383920 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.383926 5030 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2802b231-5ab0-43cd-85f1-c6c4c68c331a-metrics-certs\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:40.521125 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:40.521087 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:14:41.061369 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:41.061333 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" Oct 29 23:14:41.067765 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:41.067731 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4" event={"ID":"2802b231-5ab0-43cd-85f1-c6c4c68c331a","Type":"ContainerDied","Data":"d079ff1bdae3bc297dd28d49fd0dcfa89ab732b14e21dc1057ab23e1ab2b6caa"} Oct 29 23:14:41.067948 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:41.067819 5030 scope.go:122] "RemoveContainer" containerID="2865294ecd2a0ba5d564c56f3dfab4fb520f074021ba1e3a30c5fdfb71460d4e" Oct 29 23:14:41.122495 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:41.122465 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4"] Oct 29 23:14:41.135096 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:41.135071 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-wnnx4"] Oct 29 23:14:43.056874 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:43.049243 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2802b231-5ab0-43cd-85f1-c6c4c68c331a" path="/var/lib/kubelet/pods/2802b231-5ab0-43cd-85f1-c6c4c68c331a/volumes" Oct 29 23:14:47.396651 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.396620 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv"] Oct 29 23:14:47.397201 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.397185 5030 cpu_manager.go:414] "containerMap: removing container" podUID="2802b231-5ab0-43cd-85f1-c6c4c68c331a" containerName="maas-api" Oct 29 23:14:47.397246 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.397202 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2802b231-5ab0-43cd-85f1-c6c4c68c331a" containerName="maas-api" Oct 29 23:14:47.397246 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.397234 5030 cpu_manager.go:414] "containerMap: removing container" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:14:47.397246 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.397239 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:14:47.397352 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.397340 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f361422-5141-4f60-833a-e0ae904c5a70" containerName="cleanup" Oct 29 23:14:47.397352 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.397351 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="2802b231-5ab0-43cd-85f1-c6c4c68c331a" containerName="maas-api" Oct 29 23:14:47.402086 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.402071 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.420175 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.420147 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv"] Oct 29 23:14:47.429140 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.429115 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 29 23:14:47.429301 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.429172 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjz6s\"" Oct 29 23:14:47.477634 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.477600 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.477770 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.477661 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pj6g\" (UniqueName: \"kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.477814 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.477774 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.578583 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.578551 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pj6g\" (UniqueName: \"kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.578747 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.578591 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.578789 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.578758 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.579295 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.579273 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.581333 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.581308 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.598827 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.598797 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pj6g\" (UniqueName: \"kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g\") pod \"maas-api-revoke-sub-f03629db-fwzjv\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:47.716645 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:47.716624 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:48.015698 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:48.015675 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv"] Oct 29 23:14:48.018024 ip-10-0-136-228 kubenswrapper[5030]: W1029 23:14:48.017994 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3ec120_cbcb_438b_8c98_2e062681fc63.slice/crio-dd9f1cf350f02cc49ae61c018c77ba24d92aa46f42db9549598b87704e6b694d WatchSource:0}: Error finding container dd9f1cf350f02cc49ae61c018c77ba24d92aa46f42db9549598b87704e6b694d: Status 404 returned error can't find the container with id dd9f1cf350f02cc49ae61c018c77ba24d92aa46f42db9549598b87704e6b694d Oct 29 23:14:48.162823 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:48.162789 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" event={"ID":"8d3ec120-cbcb-438b-8c98-2e062681fc63","Type":"ContainerStarted","Data":"dd9f1cf350f02cc49ae61c018c77ba24d92aa46f42db9549598b87704e6b694d"} Oct 29 23:14:49.169893 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:49.169784 5030 generic.go:356] "Generic (PLEG): container finished" podID="8d3ec120-cbcb-438b-8c98-2e062681fc63" containerID="17dfdcfaa369be559045bb591696093da2deef6a8e477287c8787500de2d2b48" exitCode=0 Oct 29 23:14:49.169893 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:49.169878 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" event={"ID":"8d3ec120-cbcb-438b-8c98-2e062681fc63","Type":"ContainerDied","Data":"17dfdcfaa369be559045bb591696093da2deef6a8e477287c8787500de2d2b48"} Oct 29 23:14:50.316924 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.316904 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:50.406997 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.406972 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca\") pod \"8d3ec120-cbcb-438b-8c98-2e062681fc63\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " Oct 29 23:14:50.407140 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.407010 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g\" (UniqueName: \"kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g\") pod \"8d3ec120-cbcb-438b-8c98-2e062681fc63\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " Oct 29 23:14:50.407140 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.407060 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca\") pod \"8d3ec120-cbcb-438b-8c98-2e062681fc63\" (UID: \"8d3ec120-cbcb-438b-8c98-2e062681fc63\") " Oct 29 23:14:50.407326 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.407295 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca" pod "8d3ec120-cbcb-438b-8c98-2e062681fc63" (UID: "8d3ec120-cbcb-438b-8c98-2e062681fc63"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:14:50.409360 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.409335 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca" pod "8d3ec120-cbcb-438b-8c98-2e062681fc63" (UID: "8d3ec120-cbcb-438b-8c98-2e062681fc63"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:14:50.409360 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.409336 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g" pod "8d3ec120-cbcb-438b-8c98-2e062681fc63" (UID: "8d3ec120-cbcb-438b-8c98-2e062681fc63"). InnerVolumeSpecName "kube-api-access-9pj6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:14:50.508400 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.508374 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-openshift-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:50.508400 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.508400 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pj6g\" (UniqueName: \"kubernetes.io/projected/8d3ec120-cbcb-438b-8c98-2e062681fc63-kube-api-access-9pj6g\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:50.508556 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:50.508411 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d3ec120-cbcb-438b-8c98-2e062681fc63-maas-api-xks-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:14:51.182457 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:51.182419 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" event={"ID":"8d3ec120-cbcb-438b-8c98-2e062681fc63","Type":"ContainerDied","Data":"dd9f1cf350f02cc49ae61c018c77ba24d92aa46f42db9549598b87704e6b694d"} Oct 29 23:14:51.182457 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:51.182457 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-fwzjv" Oct 29 23:14:51.182653 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:51.182467 5030 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd9f1cf350f02cc49ae61c018c77ba24d92aa46f42db9549598b87704e6b694d" Oct 29 23:14:54.501231 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:54.501200 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:14:59.377863 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:59.377506 5030 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:14:59.377863 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:14:59.377588 5030 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rt2xq" podUID="fa079846-a34e-4362-b007-8a85af7a852a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:15:00.139459 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.139429 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7"] Oct 29 23:15:00.140335 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.140315 5030 cpu_manager.go:414] "containerMap: removing container" podUID="8d3ec120-cbcb-438b-8c98-2e062681fc63" containerName="revoke-keys" Oct 29 23:15:00.140394 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.140341 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d3ec120-cbcb-438b-8c98-2e062681fc63" containerName="revoke-keys" Oct 29 23:15:00.140513 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.140503 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d3ec120-cbcb-438b-8c98-2e062681fc63" containerName="revoke-keys" Oct 29 23:15:00.144121 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.144101 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.161929 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.161902 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7"] Oct 29 23:15:00.170954 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.170929 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjz6s\"" Oct 29 23:15:00.246005 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.245972 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888595-lphd7\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.246184 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.246145 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcznc\" (UniqueName: \"kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc\") pod \"maas-api-key-cleanup-29888595-lphd7\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.347663 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.347638 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcznc\" (UniqueName: \"kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc\") pod \"maas-api-key-cleanup-29888595-lphd7\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.347869 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.347711 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888595-lphd7\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.348321 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.348301 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888595-lphd7\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.367909 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.367886 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcznc\" (UniqueName: \"kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc\") pod \"maas-api-key-cleanup-29888595-lphd7\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.458899 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.458809 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:00.758966 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:00.758936 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7"] Oct 29 23:15:00.765452 ip-10-0-136-228 kubenswrapper[5030]: W1029 23:15:00.765424 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb79756_eabd_44a4_83c7_09d7730d7802.slice/crio-6a4634a58b0eac4843430daf98503a397a65ce3cfaf17efc8c7add2ac195d8ef WatchSource:0}: Error finding container 6a4634a58b0eac4843430daf98503a397a65ce3cfaf17efc8c7add2ac195d8ef: Status 404 returned error can't find the container with id 6a4634a58b0eac4843430daf98503a397a65ce3cfaf17efc8c7add2ac195d8ef Oct 29 23:15:01.257650 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:01.257617 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" event={"ID":"ddb79756-eabd-44a4-83c7-09d7730d7802","Type":"ContainerStarted","Data":"9b649ce9c329fca74f9f4a3679a5dab1d86e451c36784ae55ccfa9dc06740f10"} Oct 29 23:15:01.257828 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:01.257672 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" event={"ID":"ddb79756-eabd-44a4-83c7-09d7730d7802","Type":"ContainerStarted","Data":"6a4634a58b0eac4843430daf98503a397a65ce3cfaf17efc8c7add2ac195d8ef"} Oct 29 23:15:01.297635 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:01.297589 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" podStartSLOduration=1.2975761860000001 podStartE2EDuration="1.297576186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:15:01.296480296 +0000 UTC m=+2498.409125883" watchObservedRunningTime="2026-10-29 23:15:01.297576186 +0000 UTC m=+2498.410221774" Oct 29 23:15:02.265689 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:02.265655 5030 generic.go:356] "Generic (PLEG): container finished" podID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerID="9b649ce9c329fca74f9f4a3679a5dab1d86e451c36784ae55ccfa9dc06740f10" exitCode=60 Oct 29 23:15:02.266094 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:02.265714 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" event={"ID":"ddb79756-eabd-44a4-83c7-09d7730d7802","Type":"ContainerDied","Data":"9b649ce9c329fca74f9f4a3679a5dab1d86e451c36784ae55ccfa9dc06740f10"} Oct 29 23:15:02.266094 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:02.265935 5030 scope.go:122] "RemoveContainer" containerID="9b649ce9c329fca74f9f4a3679a5dab1d86e451c36784ae55ccfa9dc06740f10" Oct 29 23:15:03.273063 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:03.273029 5030 generic.go:356] "Generic (PLEG): container finished" podID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerID="172b489b64809532b76ccd2f8587abbac322e0f00a035bb9972a8bbe2da8e9de" exitCode=60 Oct 29 23:15:03.273515 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:03.273094 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" event={"ID":"ddb79756-eabd-44a4-83c7-09d7730d7802","Type":"ContainerDied","Data":"172b489b64809532b76ccd2f8587abbac322e0f00a035bb9972a8bbe2da8e9de"} Oct 29 23:15:03.273515 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:03.273162 5030 scope.go:122] "RemoveContainer" containerID="9b649ce9c329fca74f9f4a3679a5dab1d86e451c36784ae55ccfa9dc06740f10" Oct 29 23:15:03.273515 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:03.273385 5030 scope.go:122] "RemoveContainer" containerID="172b489b64809532b76ccd2f8587abbac322e0f00a035bb9972a8bbe2da8e9de" Oct 29 23:15:03.273633 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:15:03.273615 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888595-lphd7_odh-ai-gateway-infra(ddb79756-eabd-44a4-83c7-09d7730d7802)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" Oct 29 23:15:04.280572 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:04.280543 5030 scope.go:122] "RemoveContainer" containerID="172b489b64809532b76ccd2f8587abbac322e0f00a035bb9972a8bbe2da8e9de" Oct 29 23:15:04.280984 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:15:04.280796 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888595-lphd7_odh-ai-gateway-infra(ddb79756-eabd-44a4-83c7-09d7730d7802)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" Oct 29 23:15:05.794740 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:15:05.794705 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:15:17.037923 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:17.037888 5030 scope.go:122] "RemoveContainer" containerID="172b489b64809532b76ccd2f8587abbac322e0f00a035bb9972a8bbe2da8e9de" Oct 29 23:15:18.369461 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:18.369430 5030 generic.go:356] "Generic (PLEG): container finished" podID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerID="49ca2c9b8da0cced0c49436258e728ad1786a77f249d11f27a2465b4b945b625" exitCode=60 Oct 29 23:15:18.369923 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:18.369508 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" event={"ID":"ddb79756-eabd-44a4-83c7-09d7730d7802","Type":"ContainerDied","Data":"49ca2c9b8da0cced0c49436258e728ad1786a77f249d11f27a2465b4b945b625"} Oct 29 23:15:18.369923 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:18.369570 5030 scope.go:122] "RemoveContainer" containerID="172b489b64809532b76ccd2f8587abbac322e0f00a035bb9972a8bbe2da8e9de" Oct 29 23:15:18.370055 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:18.369994 5030 scope.go:122] "RemoveContainer" containerID="49ca2c9b8da0cced0c49436258e728ad1786a77f249d11f27a2465b4b945b625" Oct 29 23:15:18.370331 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:15:18.370304 5030 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29888595-lphd7_odh-ai-gateway-infra(ddb79756-eabd-44a4-83c7-09d7730d7802)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" Oct 29 23:15:19.427377 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.427347 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7"] Oct 29 23:15:19.581909 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.581884 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:19.631886 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.631833 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca\") pod \"ddb79756-eabd-44a4-83c7-09d7730d7802\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " Oct 29 23:15:19.632008 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.631924 5030 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc\" (UniqueName: \"kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc\") pod \"ddb79756-eabd-44a4-83c7-09d7730d7802\" (UID: \"ddb79756-eabd-44a4-83c7-09d7730d7802\") " Oct 29 23:15:19.632141 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.632120 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca" pod "ddb79756-eabd-44a4-83c7-09d7730d7802" (UID: "ddb79756-eabd-44a4-83c7-09d7730d7802"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:15:19.632244 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.632227 5030 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb79756-eabd-44a4-83c7-09d7730d7802-maas-api-service-ca\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:15:19.634210 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.634189 5030 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc" pod "ddb79756-eabd-44a4-83c7-09d7730d7802" (UID: "ddb79756-eabd-44a4-83c7-09d7730d7802"). InnerVolumeSpecName "kube-api-access-vcznc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:15:19.733052 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:19.733026 5030 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcznc\" (UniqueName: \"kubernetes.io/projected/ddb79756-eabd-44a4-83c7-09d7730d7802-kube-api-access-vcznc\") on node \"ip-10-0-136-228.ec2.internal\" DevicePath \"\"" Oct 29 23:15:20.382936 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:20.382901 5030 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" Oct 29 23:15:20.382936 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:20.382910 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7" event={"ID":"ddb79756-eabd-44a4-83c7-09d7730d7802","Type":"ContainerDied","Data":"6a4634a58b0eac4843430daf98503a397a65ce3cfaf17efc8c7add2ac195d8ef"} Oct 29 23:15:20.383200 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:20.382984 5030 scope.go:122] "RemoveContainer" containerID="49ca2c9b8da0cced0c49436258e728ad1786a77f249d11f27a2465b4b945b625" Oct 29 23:15:20.434767 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:20.434741 5030 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7"] Oct 29 23:15:20.447153 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:20.447126 5030 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888595-lphd7"] Oct 29 23:15:21.045097 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:21.045064 5030 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" path="/var/lib/kubelet/pods/ddb79756-eabd-44a4-83c7-09d7730d7802/volumes" Oct 29 23:15:28.218792 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:15:28.218755 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spfbm"] Oct 29 23:15:35.794274 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:15:35.794239 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:16:05.795461 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:16:05.795416 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:16:09.909720 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:09.909690 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ngv8z_4b14f25f-dafa-4156-aa19-fd2aad88ec80/manager/0.log" Oct 29 23:16:10.410291 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:10.410264 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tnntd_65a8235c-4125-4ca5-82fb-3f61078c8dbd/server/0.log" Oct 29 23:16:11.536485 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:11.536458 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-675b698459-wwm4m_0cc1c950-de4e-4ce7-8ab5-20415c1e62ed/copy-manifests/0.log" Oct 29 23:16:11.564188 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:11.564163 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-675b698459-wwm4m_0cc1c950-de4e-4ce7-8ab5-20415c1e62ed/manager/0.log" Oct 29 23:16:11.762709 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:11.762682 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-w4nsp_9a233e1b-cf6f-42a7-9141-5aded41232ce/proxy/0.log" Oct 29 23:16:14.247373 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:14.247345 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-fwzjv_8d3ec120-cbcb-438b-8c98-2e062681fc63/revoke-keys/0.log" Oct 29 23:16:14.440678 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:14.440648 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-gjp6d_bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97/postgres/0.log" Oct 29 23:16:16.195639 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:16.195611 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-6fh2m_ce6d1137-ebfd-454a-8c9b-022babc856bf/manager/0.log" Oct 29 23:16:16.389050 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:16.389022 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-vrxt9_50f1affa-2d7d-49a6-b925-eaaae50edbd3/manager/0.log" Oct 29 23:16:16.574716 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:16.574690 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-6md66_3f1c6a88-9498-46a4-9df4-3916e42520e6/kuadrant-console-plugin/0.log" Oct 29 23:16:16.779578 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:16.779552 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-kwjnc_74ecea0c-0833-4168-8e4b-35dc438ad4de/manager/0.log" Oct 29 23:16:17.148118 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:17.148087 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-spfbm_7773fbc0-77b8-4d48-94f5-7c3e18de77b3/limitador/0.log" Oct 29 23:16:17.332560 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:17.332527 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-cqkmp_8c4e0ed4-a353-41ca-bff4-eaef1d54137b/manager/0.log" Oct 29 23:16:19.060024 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:19.060000 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-8m9xv_b6326e02-98c1-4554-8590-11a71357b8c7/istio-proxy/0.log" Oct 29 23:16:19.667417 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:19.667392 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-gsfnp_092a3e1c-2934-486f-8fd9-35cc2e605a01/istio-proxy/0.log" Oct 29 23:16:20.268114 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:20.268086 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rt2xq_fa079846-a34e-4362-b007-8a85af7a852a/istio-proxy/1.log" Oct 29 23:16:21.267709 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:21.267681 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6bb75897-sr2f4_e9fb3bd9-7312-4616-a8db-6e51b56d78a2/router/0.log" Oct 29 23:16:21.708577 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:21.708503 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rt2xq_fa079846-a34e-4362-b007-8a85af7a852a/istio-proxy/0.log" Oct 29 23:16:34.156519 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:34.156492 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ngv8z_4b14f25f-dafa-4156-aa19-fd2aad88ec80/manager/0.log" Oct 29 23:16:34.630090 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:34.630063 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-tnntd_65a8235c-4125-4ca5-82fb-3f61078c8dbd/server/0.log" Oct 29 23:16:35.794482 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:16:35.794445 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:16:35.822588 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:35.822565 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-675b698459-wwm4m_0cc1c950-de4e-4ce7-8ab5-20415c1e62ed/copy-manifests/0.log" Oct 29 23:16:35.852784 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:35.852754 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-675b698459-wwm4m_0cc1c950-de4e-4ce7-8ab5-20415c1e62ed/manager/0.log" Oct 29 23:16:36.036062 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:36.036034 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-w4nsp_9a233e1b-cf6f-42a7-9141-5aded41232ce/proxy/0.log" Oct 29 23:16:38.148446 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:38.148419 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-fwzjv_8d3ec120-cbcb-438b-8c98-2e062681fc63/revoke-keys/0.log" Oct 29 23:16:38.333621 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:38.333590 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-gjp6d_bc5d07e1-6c1e-4c80-84a3-d5d8bfb60c97/postgres/0.log" Oct 29 23:16:39.712717 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:39.712686 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-6fh2m_ce6d1137-ebfd-454a-8c9b-022babc856bf/manager/0.log" Oct 29 23:16:39.909298 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:39.909250 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-vrxt9_50f1affa-2d7d-49a6-b925-eaaae50edbd3/manager/0.log" Oct 29 23:16:40.087872 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:40.087850 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-6md66_3f1c6a88-9498-46a4-9df4-3916e42520e6/kuadrant-console-plugin/0.log" Oct 29 23:16:40.287443 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:40.287417 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-kwjnc_74ecea0c-0833-4168-8e4b-35dc438ad4de/manager/0.log" Oct 29 23:16:40.574077 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:40.574050 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-spfbm_7773fbc0-77b8-4d48-94f5-7c3e18de77b3/limitador/0.log" Oct 29 23:16:40.767099 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:40.767070 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-cqkmp_8c4e0ed4-a353-41ca-bff4-eaef1d54137b/manager/0.log" Oct 29 23:16:42.374369 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:42.374337 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-8m9xv_b6326e02-98c1-4554-8590-11a71357b8c7/istio-proxy/0.log" Oct 29 23:16:43.001256 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:43.001228 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-gsfnp_092a3e1c-2934-486f-8fd9-35cc2e605a01/istio-proxy/0.log" Oct 29 23:16:43.682344 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:43.682315 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rt2xq_fa079846-a34e-4362-b007-8a85af7a852a/istio-proxy/1.log" Oct 29 23:16:44.690969 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:16:44.690943 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6bb75897-sr2f4_e9fb3bd9-7312-4616-a8db-6e51b56d78a2/router/0.log" Oct 29 23:17:05.794381 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:17:05.794338 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:17:35.793748 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:17:35.793660 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:18:05.793991 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:18:05.793956 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:18:18.956701 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:18.956671 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-65b9p_3f37d2fc-3cf7-4105-9c73-bf532a71af91/global-pull-secret-syncer/0.log" Oct 29 23:18:18.957680 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:18.957660 5030 ???:1] "http: TLS handshake error from 10.0.136.228:48226: EOF" Oct 29 23:18:19.461277 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:19.461241 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-6z7h7_230e7bac-638f-4478-989a-abad8f7213bd/connection-checker/0.log" Oct 29 23:18:19.580532 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:19.580508 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-jlpb2_983a2044-473f-48be-8e73-5da2ed82aaf4/connection-checker/0.log" Oct 29 23:18:19.706639 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:19.706615 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-xtkwb_4a9b3ef6-bbba-450a-a8d5-b4ad21a2fe20/connection-checker/0.log" Oct 29 23:18:20.138797 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:20.138771 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-skdsx_e5092efe-96fd-4aec-99f1-a2c07c84ac7f/konnectivity-agent/0.log" Oct 29 23:18:20.403580 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:20.403515 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-228.ec2.internal_2e2359a413d1cf6872204355c65cbec7/haproxy/0.log" Oct 29 23:18:25.259348 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:25.259323 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:18:25.261386 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:25.261367 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:18:26.577389 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:26.577361 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-6fh2m_ce6d1137-ebfd-454a-8c9b-022babc856bf/manager/0.log" Oct 29 23:18:26.694968 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:26.694943 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-vrxt9_50f1affa-2d7d-49a6-b925-eaaae50edbd3/manager/0.log" Oct 29 23:18:26.818975 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:26.818950 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-6md66_3f1c6a88-9498-46a4-9df4-3916e42520e6/kuadrant-console-plugin/0.log" Oct 29 23:18:26.960034 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:26.959970 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-kwjnc_74ecea0c-0833-4168-8e4b-35dc438ad4de/manager/0.log" Oct 29 23:18:27.150485 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:27.150462 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-spfbm_7773fbc0-77b8-4d48-94f5-7c3e18de77b3/limitador/0.log" Oct 29 23:18:27.295356 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:27.295322 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-cqkmp_8c4e0ed4-a353-41ca-bff4-eaef1d54137b/manager/0.log" Oct 29 23:18:28.581425 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.581397 5030 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn"] Oct 29 23:18:28.582472 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582447 5030 cpu_manager.go:414] "containerMap: removing container" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.582472 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582469 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.582640 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582485 5030 cpu_manager.go:414] "containerMap: removing container" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.582640 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582490 5030 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.582640 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582596 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.582640 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582604 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.582640 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.582613 5030 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddb79756-eabd-44a4-83c7-09d7730d7802" containerName="cleanup" Oct 29 23:18:28.585597 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.585583 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.601224 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.601204 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbmpv\"/\"kube-root-ca.crt\"" Oct 29 23:18:28.604611 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.604586 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn"] Oct 29 23:18:28.613803 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.613779 5030 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbmpv\"/\"openshift-service-ca.crt\"" Oct 29 23:18:28.683999 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.683976 5030 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rbmpv\"/\"default-dockercfg-wghv7\"" Oct 29 23:18:28.692478 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.692444 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-lib-modules\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.692564 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.692484 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-sys\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.692622 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.692606 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-proc\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.692675 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.692663 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-podres\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.692785 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.692767 5030 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2z84\" (UniqueName: \"kubernetes.io/projected/0a082b98-9126-4615-8983-94ba2b5d6396-kube-api-access-w2z84\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794211 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794176 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2z84\" (UniqueName: \"kubernetes.io/projected/0a082b98-9126-4615-8983-94ba2b5d6396-kube-api-access-w2z84\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794342 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794239 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-lib-modules\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794342 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794265 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-sys\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794437 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794341 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-proc\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794437 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794392 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-lib-modules\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794437 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794406 5030 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-podres\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794437 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794430 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-sys\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794574 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794462 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-podres\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.794574 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.794498 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0a082b98-9126-4615-8983-94ba2b5d6396-proc\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.814387 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.814358 5030 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2z84\" (UniqueName: \"kubernetes.io/projected/0a082b98-9126-4615-8983-94ba2b5d6396-kube-api-access-w2z84\") pod \"perf-node-gather-daemonset-qlrsn\" (UID: \"0a082b98-9126-4615-8983-94ba2b5d6396\") " pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.851234 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.851154 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/alertmanager/0.log" Oct 29 23:18:28.898211 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.898181 5030 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:28.969343 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:28.969319 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/config-reloader/0.log" Oct 29 23:18:29.081595 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.081570 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/kube-rbac-proxy-web/0.log" Oct 29 23:18:29.200319 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.200294 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/kube-rbac-proxy/0.log" Oct 29 23:18:29.201126 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.201104 5030 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn"] Oct 29 23:18:29.203222 ip-10-0-136-228 kubenswrapper[5030]: W1029 23:18:29.203192 5030 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0a082b98_9126_4615_8983_94ba2b5d6396.slice/crio-5daf88150a69fcb1336283895e38db157db7d82ac1e2d60b3ab0ab7fa9c8217a WatchSource:0}: Error finding container 5daf88150a69fcb1336283895e38db157db7d82ac1e2d60b3ab0ab7fa9c8217a: Status 404 returned error can't find the container with id 5daf88150a69fcb1336283895e38db157db7d82ac1e2d60b3ab0ab7fa9c8217a Oct 29 23:18:29.315176 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.315133 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/kube-rbac-proxy-metric/0.log" Oct 29 23:18:29.427519 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.427496 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/prom-label-proxy/0.log" Oct 29 23:18:29.432963 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.432939 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" event={"ID":"0a082b98-9126-4615-8983-94ba2b5d6396","Type":"ContainerStarted","Data":"af16b6e427c6fc04880f14ccab8537bccd2eb68f954fbcad78ded01472f10206"} Oct 29 23:18:29.433066 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.432987 5030 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" event={"ID":"0a082b98-9126-4615-8983-94ba2b5d6396","Type":"ContainerStarted","Data":"5daf88150a69fcb1336283895e38db157db7d82ac1e2d60b3ab0ab7fa9c8217a"} Oct 29 23:18:29.433066 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.433040 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:29.476529 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.476481 5030 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" podStartSLOduration=1.476468666 podStartE2EDuration="1.476468666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:18:29.474661903 +0000 UTC m=+2706.587307491" watchObservedRunningTime="2026-10-29 23:18:29.476468666 +0000 UTC m=+2706.589114253" Oct 29 23:18:29.541544 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.541526 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ff86b1f-1358-4900-becc-5ab1f332ed9a/init-config-reloader/0.log" Oct 29 23:18:29.670733 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.670710 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-lmjqp_2ce7d6cc-47fa-4a6a-ba52-5a23efd31019/cluster-monitoring-operator/0.log" Oct 29 23:18:29.780920 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.780856 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-75qms_48d3d9f0-9c5e-4296-bb37-2bee105913c7/kube-state-metrics/0.log" Oct 29 23:18:29.888785 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:29.888764 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-75qms_48d3d9f0-9c5e-4296-bb37-2bee105913c7/kube-rbac-proxy-main/0.log" Oct 29 23:18:30.002374 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:30.002349 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-75qms_48d3d9f0-9c5e-4296-bb37-2bee105913c7/kube-rbac-proxy-self/0.log" Oct 29 23:18:30.119620 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:30.119555 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-77dd54f899-2qj6z_154474f2-5e45-48f6-9a83-2aa3866e2328/metrics-server/0.log" Oct 29 23:18:30.233148 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:30.233115 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-blqdr_d39741d3-5f7e-4082-9bc3-a1c5d38447da/monitoring-plugin/0.log" Oct 29 23:18:31.074539 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.074516 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v7bzc_03802761-7c3b-41b3-a7c5-81f4f225527e/node-exporter/0.log" Oct 29 23:18:31.185407 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.185382 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v7bzc_03802761-7c3b-41b3-a7c5-81f4f225527e/kube-rbac-proxy/0.log" Oct 29 23:18:31.297000 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.296970 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v7bzc_03802761-7c3b-41b3-a7c5-81f4f225527e/init-textfile/0.log" Oct 29 23:18:31.412689 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.412623 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-6gbxb_0fd020fd-242a-4233-885d-c120ad267ce2/kube-rbac-proxy-main/0.log" Oct 29 23:18:31.525135 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.525109 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-6gbxb_0fd020fd-242a-4233-885d-c120ad267ce2/kube-rbac-proxy-self/0.log" Oct 29 23:18:31.637074 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.637045 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-6gbxb_0fd020fd-242a-4233-885d-c120ad267ce2/openshift-state-metrics/0.log" Oct 29 23:18:31.758800 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.758775 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/prometheus/0.log" Oct 29 23:18:31.871078 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.871047 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/config-reloader/0.log" Oct 29 23:18:31.984792 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:31.984769 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/thanos-sidecar/0.log" Oct 29 23:18:32.098630 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.098565 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/kube-rbac-proxy-web/0.log" Oct 29 23:18:32.212654 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.212632 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/kube-rbac-proxy/0.log" Oct 29 23:18:32.326908 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.326880 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/kube-rbac-proxy-thanos/0.log" Oct 29 23:18:32.442315 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.442252 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6e8774a7-6423-4810-933a-705d058e884d/init-config-reloader/0.log" Oct 29 23:18:32.561558 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.561533 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-pnvfl_b3d52fe7-a39c-43de-bf41-ee90256e755a/prometheus-operator/0.log" Oct 29 23:18:32.671537 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.671512 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-pnvfl_b3d52fe7-a39c-43de-bf41-ee90256e755a/kube-rbac-proxy/0.log" Oct 29 23:18:32.787882 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.787832 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-hmmgp_07cb55c7-98cf-4c00-8e65-11a26ad2eca7/prometheus-operator-admission-webhook/0.log" Oct 29 23:18:32.905901 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:32.905875 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bd5db97b-9zfb8_b6e248fe-a23c-46ed-b15d-6b318912801d/telemeter-client/0.log" Oct 29 23:18:33.019264 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.019242 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bd5db97b-9zfb8_b6e248fe-a23c-46ed-b15d-6b318912801d/reload/0.log" Oct 29 23:18:33.132995 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.132917 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bd5db97b-9zfb8_b6e248fe-a23c-46ed-b15d-6b318912801d/kube-rbac-proxy/0.log" Oct 29 23:18:33.250608 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.250585 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7986f44ccf-p28v2_ddd8a66a-5b7f-436e-923f-3a69a9ee546f/thanos-query/0.log" Oct 29 23:18:33.362764 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.362741 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7986f44ccf-p28v2_ddd8a66a-5b7f-436e-923f-3a69a9ee546f/kube-rbac-proxy-web/0.log" Oct 29 23:18:33.478640 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.478613 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7986f44ccf-p28v2_ddd8a66a-5b7f-436e-923f-3a69a9ee546f/kube-rbac-proxy/0.log" Oct 29 23:18:33.590610 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.590576 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7986f44ccf-p28v2_ddd8a66a-5b7f-436e-923f-3a69a9ee546f/prom-label-proxy/0.log" Oct 29 23:18:33.702603 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.702552 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7986f44ccf-p28v2_ddd8a66a-5b7f-436e-923f-3a69a9ee546f/kube-rbac-proxy-rules/0.log" Oct 29 23:18:33.816878 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:33.816788 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7986f44ccf-p28v2_ddd8a66a-5b7f-436e-923f-3a69a9ee546f/kube-rbac-proxy-metrics/0.log" Oct 29 23:18:35.321560 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:35.321532 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-w564k_bba55700-7326-482d-919f-02f3eff316a1/networking-console-plugin/0.log" Oct 29 23:18:35.449739 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:35.449711 5030 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rbmpv/perf-node-gather-daemonset-qlrsn" Oct 29 23:18:35.794344 ip-10-0-136-228 kubenswrapper[5030]: E1029 23:18:35.794308 5030 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tdvz7" containerName="extractor" Oct 29 23:18:36.523037 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:36.522984 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/2.log" Oct 29 23:18:36.528114 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:36.528083 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-m855q_5d83e7cd-0fff-4ad5-9258-ef3af8a7a2cb/console-operator/3.log" Oct 29 23:18:37.518828 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:37.518803 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7595bd47b-dpvnl_a90bfb0c-6f10-431f-8686-3a8fdbf80718/console/0.log" Oct 29 23:18:37.656395 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:37.656367 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-r7wrt_3897b32b-ba8f-41b7-bc91-db7901bb8335/download-server/0.log" Oct 29 23:18:39.107125 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:39.107091 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-829vk_86b59e01-ac2f-4f6d-bc4c-fcbc59afe2a9/volume-data-source-validator/0.log" Oct 29 23:18:41.951335 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:41.951304 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vml7h_efa7fc2d-d350-4aba-977b-32868a6e3fe7/dns/0.log" Oct 29 23:18:42.060743 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:42.060714 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vml7h_efa7fc2d-d350-4aba-977b-32868a6e3fe7/kube-rbac-proxy/0.log" Oct 29 23:18:42.177085 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:42.177056 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9655m_1ac5c5e2-a666-4cbf-9a73-677929dc640e/dns-node-resolver/0.log" Oct 29 23:18:43.109097 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.109070 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-8m9xv_b6326e02-98c1-4554-8590-11a71357b8c7/istio-proxy/0.log" Oct 29 23:18:43.301432 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.301404 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-gsfnp_092a3e1c-2934-486f-8fd9-35cc2e605a01/istio-proxy/0.log" Oct 29 23:18:43.489061 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.489036 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68487bc74d-jbpms_fe2799d4-b34d-47a7-acb8-80ffbf92b66b/registry/0.log" Oct 29 23:18:43.490184 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.490160 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rt2xq_fa079846-a34e-4362-b007-8a85af7a852a/istio-proxy/0.log" Oct 29 23:18:43.491823 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.491805 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rt2xq_fa079846-a34e-4362-b007-8a85af7a852a/istio-proxy/1.log" Oct 29 23:18:43.725169 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.725140 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gkr9j_c7881bd2-9f24-4c58-9a77-68cc091ee9ab/node-ca/0.log" Oct 29 23:18:43.810366 ip-10-0-136-228 kubenswrapper[5030]: I1029 23:18:43.810303 5030 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6bb75897-sr2f4_e9fb3bd9-7312-4616-a8db-6e51b56d78a2/router/0.log"