Oct 30 21:53:48.295721 ip-10-0-138-166 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:53:48.665762 ip-10-0-138-166 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:53:48.665762 ip-10-0-138-166 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:53:48.665762 ip-10-0-138-166 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:53:48.665762 ip-10-0-138-166 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:53:48.670611 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670525    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.670611 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670606    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.670611 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670611    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.670611 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670614    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.670611 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670617    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670620    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670623    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670626    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670629    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670632    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670635    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670638    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670640    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670643    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670646    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670648    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670651    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670654    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670656    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670659    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670665    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670667    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670670    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670672    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.670795 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670675    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670678    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670680    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670683    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670685    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670688    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670692    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670697    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670700    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670703    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670705    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670708    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670710    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670713    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670716    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670718    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670721    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670724    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670726    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670729    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.671360 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670731    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670733    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670736    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670738    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670741    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670743    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670747    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670749    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670752    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670754    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670757    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670759    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670762    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670764    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670767    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670769    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670771    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670774    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670776    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.671853 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670779    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670782    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670784    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670787    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670790    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670793    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670795    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670798    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670801    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670804    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670807    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670810    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670812    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670815    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670818    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670821    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670823    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670826    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670828    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.672328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670831    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670835    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670838    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670841    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670843    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670846    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670848    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670850    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670853    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670856    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670858    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670861    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670863    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670868    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670872    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670875    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670878    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670881    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670884    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.672791 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670886    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670889    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670892    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670895    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.670897    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671693    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671702    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671706    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671710    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671713    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671717    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671720    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671724    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671726    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671729    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671732    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671735    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671737    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671740    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671743    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.673229 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671745    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671748    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671750    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671753    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671755    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671758    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671760    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671763    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671766    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671768    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671770    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671773    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671775    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671778    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671781    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671783    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671787    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671789    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671791    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671794    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.673735 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671796    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671799    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671801    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671803    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671806    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671808    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671811    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671813    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671816    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671819    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671821    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671823    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671826    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671828    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671830    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671833    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671835    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671838    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671842    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.674221 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671844    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671847    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671849    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671852    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671855    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671857    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671860    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671863    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671866    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671869    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671871    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671874    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671876    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671878    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671881    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671884    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671886    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671889    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671891    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671894    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.674691 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671896    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671899    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671901    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671904    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671906    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671909    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671911    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671914    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671917    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671919    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671921    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671924    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671927    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671929    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671932    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671935    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671937    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671940    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671942    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.675202 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671945    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671948    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671951    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671955    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671958    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671961    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671963    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671966    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671968    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671970    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671973    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671975    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.671978    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672474    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672480    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672483    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672486    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672489    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672491    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.675681 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672494    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672496    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672499    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672501    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672504    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672506    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672509    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672512    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672514    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672516    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672519    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672522    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672524    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672528    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672530    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672533    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672535    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672538    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672540    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672542    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.676129 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672545    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672547    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672550    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672552    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672555    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672557    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672560    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672562    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672564    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672568    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672570    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672573    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672575    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672577    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672580    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672582    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672585    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672588    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672590    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.676629 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672593    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672595    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672598    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672601    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672603    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672605    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672608    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672610    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672613    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672617    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672620    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672623    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672626    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672629    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672631    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672635    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672637    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672640    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672642    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.677065 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672644    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672647    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672649    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672652    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672655    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672657    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672660    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672662    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672665    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672667    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672670    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672672    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672675    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672677    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672679    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672682    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672684    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672686    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672689    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672691    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.677531 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672694    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672696    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672699    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672701    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672703    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672706    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672710    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672713    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672716    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672718    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672721    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672723    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672726    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672728    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672730    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672733    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672736    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672739    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672757    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.678041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672762    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672765    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.672768    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673502    2576 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673515    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673521    2576 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673526    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673530    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673533    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673538    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673542    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673545    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673548    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673551    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673557    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673560    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673563    2576 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673565    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673568    2576 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673571    2576 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673574    2576 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673580    2576 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673585    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:53:48.678507 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673588    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673591    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673595    2576 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673599    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673602    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673605    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673608    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673611    2576 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673615    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673618    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673621    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673624    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673628    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673631    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673634    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673637    2576 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673640    2576 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673642    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673647    2576 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673651    2576 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673653    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673656    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673660    2576 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673665    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:53:48.679039 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673668    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673671    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673675    2576 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673678    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673681    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673684    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673686    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673691    2576 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673694    2576 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673697    2576 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673701    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673704    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673707    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673710    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673713    2576 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673716    2576 flags.go:64] FLAG: --help="false"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673719    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673723    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673726    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673729    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673732    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673735    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673738    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:53:48.679718 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673741    2576 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673744    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673747    2576 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673750    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673753    2576 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673756    2576 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673759    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673761    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673766    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673768    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673771    2576 flags.go:64] FLAG: --lock-file=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673774    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673777    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673780    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673786    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673788    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673793    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673796    2576 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673799    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673802    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673804    2576 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673808    2576 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673812    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673815    2576 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673819    2576 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:53:48.680257 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673822    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673825    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673828    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673831    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673834    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673837    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673840    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673847    2576 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673850    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673853    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673856    2576 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673859    2576 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673861    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673864    2576 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673867    2576 flags.go:64] FLAG: --port="10250"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673872    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673874    2576 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-06b25911203d41eed"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673878    2576 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673880    2576 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673884    2576 flags.go:64] FLAG: --register-node="true"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673886    2576 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673890    2576 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673893    2576 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673897    2576 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673900    2576 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673904    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:53:48.680855 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673907    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673910    2576 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673913    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673916    2576 flags.go:64] FLAG: --runonce="false"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673919    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673922    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673925    2576 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673928    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673930    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673934    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673937    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673940    2576 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673943    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673946    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673949    2576 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673952    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673954    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673957    2576 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673960    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673965    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673968    2576 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673972    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673976    2576 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673979    2576 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673981    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:53:48.681479 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673984    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673987    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673990    2576 flags.go:64] FLAG: --v="2"
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673994    2576 flags.go:64] FLAG: --version="false"
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.673999    2576 flags.go:64] FLAG: --vmodule=""
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.674003    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.674006    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674136    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674140    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674152    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674155    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674158    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674161    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674164    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674166    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674169    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674171    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674174    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674177    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674180    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674183    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.682060 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674185    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674187    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674190    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674192    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674195    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674197    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674200    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674203    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674206    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674208    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674210    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674213    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674215    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674218    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674220    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674224    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674226    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674229    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674231    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674234    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.682592 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674237    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674239    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674242    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674244    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674248    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674251    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674254    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674256    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674259    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674262    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674266    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674269    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674272    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674275    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674277    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674280    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674282    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674285    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674288    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.683075 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674291    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674294    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674296    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674298    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674301    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674303    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674306    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674308    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674311    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674314    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674316    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674319    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674321    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674323    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674325    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674328    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674330    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674332    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674335    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.683593 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674337    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674339    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674342    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674344    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674347    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674349    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674352    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674354    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674356    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674359    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674361    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674363    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674366    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674369    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674372    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674374    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674377    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674379    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674381    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674383    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.684080 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674386    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674388    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674391    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674406    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674409    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674411    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674414    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674416    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674419    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674422    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674424    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674426    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674429    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674431    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.674458    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674499    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.684561 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674503    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674506    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674509    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674512    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674514    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674517    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674519    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674522    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674524    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674528    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674531    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674533    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674535    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674538    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674540    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674543    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674545    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674547    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674550    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.684925 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674552    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674555    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674557    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674560    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674562    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674565    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674567    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674570    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674572    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674575    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674577    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674579    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674582    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674584    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674587    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674589    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674592    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674595    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674597    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.685370 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674600    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674602    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674604    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674608    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674610    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674613    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674616    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674618    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674620    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674623    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674625    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674627    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674630    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674633    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674635    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674638    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674640    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674643    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674646    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674648    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.685851 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674651    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674653    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674655    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674658    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674660    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674663    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674665    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674667    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674670    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674673    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674675    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674678    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674680    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674682    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674685    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674690    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674693    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674695    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674698    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674700    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.686328 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674703    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674705    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674708    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674710    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674714    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674716    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674719    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674721    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674724    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674728    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674732    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674734    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674737    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674739    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674742    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674746    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674750    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674753    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674756    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.686815 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674758    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674761    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674763    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674767    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674770    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674772    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674775    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.674777    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.681924    2576 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.681938    2576 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.682516    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.682522    2576 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682593    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682598    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682601    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682604    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682607    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682610    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682613    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682615    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.687251 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682619    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682621    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682624    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682627    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682630    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682633    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682635    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682638    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682640    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682643    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682645    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682648    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682650    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682653    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682655    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682663    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682666    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682669    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682671    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682673    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.687792 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682676    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682679    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682681    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682684    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682686    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682689    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682691    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682693    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682696    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682698    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682701    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682704    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682707    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682709    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682712    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682714    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682716    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682719    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682721    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.688249 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682724    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682726    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682729    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682731    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682734    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682736    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682738    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682741    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682743    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682751    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682754    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682756    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682759    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682761    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682764    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682767    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682769    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682771    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682774    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682776    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.688758 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682779    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682781    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682783    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682786    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682788    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682791    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682793    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682795    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682804    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682807    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682809    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682812    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682814    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682816    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682819    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682821    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682823    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682826    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682828    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682831    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.689273 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682833    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682836    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682844    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682846    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682849    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682852    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682856    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682860    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682863    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682866    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682869    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682872    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682874    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682877    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682880    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682884    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682888    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682891    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.689754 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.682893    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.682922    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683118    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683124    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683126    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683129    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683131    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683134    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683138    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683142    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683145    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683148    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683150    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683153    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683156    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683158    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.690194 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683161    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683169    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683172    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683174    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683178    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683181    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683184    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683187    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683189    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683192    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683194    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683197    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683199    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683202    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683204    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683207    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683209    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683212    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683214    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.690600 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683216    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683219    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683221    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683224    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683226    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683228    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683231    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683233    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683235    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683238    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683240    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683243    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683245    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683248    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683250    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683258    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683261    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683263    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683266    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683268    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.691042 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683271    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683273    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683275    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683278    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683280    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683283    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683285    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683287    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683290    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683292    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683295    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683297    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683299    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683302    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683304    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683307    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683309    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683311    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683314    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683316    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.691595 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683318    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683321    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683323    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683325    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683328    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683330    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683332    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683335    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683342    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683347    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683349    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683352    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683354    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683357    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683359    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683361    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683364    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683366    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683368    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.692220 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683371    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683373    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683375    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683378    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683380    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683383    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683385    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683387    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683390    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683406    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683409    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683412    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683414    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683416    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.683440    2576 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683485    2576 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:48.692961 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683489    2576 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683493    2576 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683499    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683502    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683505    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683507    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683518    2576 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683521    2576 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683523    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683526    2576 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683528    2576 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683531    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683533    2576 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683536    2576 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683538    2576 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683541    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683543    2576 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683546    2576 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683548    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:48.693537 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683551    2576 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683553    2576 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683555    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683558    2576 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683560    2576 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683563    2576 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683565    2576 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683568    2576 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683570    2576 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683573    2576 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683575    2576 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683578    2576 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683580    2576 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683583    2576 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683585    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683589    2576 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683592    2576 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683594    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683597    2576 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683601    2576 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:48.693983 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683609    2576 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683612    2576 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683614    2576 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683616    2576 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683619    2576 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683621    2576 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683624    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683626    2576 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683629    2576 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683631    2576 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683634    2576 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683636    2576 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683638    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683641    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683643    2576 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683646    2576 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683648    2576 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683650    2576 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683653    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683655    2576 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:48.694492 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683658    2576 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683660    2576 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683662    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683665    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683667    2576 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683670    2576 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683672    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683675    2576 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683677    2576 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683680    2576 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683682    2576 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683686    2576 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683688    2576 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683696    2576 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683699    2576 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683701    2576 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683704    2576 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683706    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683708    2576 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:48.695041 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683711    2576 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683713    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683716    2576 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683719    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683721    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683724    2576 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683726    2576 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683734    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683737    2576 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683739    2576 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683742    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683744    2576 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683746    2576 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683749    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683751    2576 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683753    2576 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683756    2576 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683760    2576 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683763    2576 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:48.695585 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683766    2576 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683769    2576 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683772    2576 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683775    2576 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683777    2576 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683781    2576 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683784    2576 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:48.683791    2576 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.683845    2576 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.685766    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.686649    2576 server.go:1014] "Starting client certificate rotation"
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.686754    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.687486    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.688818    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:53:48.696020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.695881    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:53:48.707270 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.707248    2576 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:53:48.710456 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.710442    2576 log.go:25] "Validated CRI v1 image API"
Oct 30 21:53:48.711277 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.711247    2576 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:53:48.716456 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.716435    2576 fs.go:135] Filesystem UUIDs: map[703f5809-d18f-477d-8c75-6df7d9e4cc21:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f1836b2b-b7b1-4d97-91b8-9b6e595017d4:/dev/nvme0n1p3]
Oct 30 21:53:48.716574 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.716455    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:53:48.721616 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.721489    2576 manager.go:217] Machine: {Timestamp:2026-10-30 21:53:48.720197938 +0000 UTC m=+0.312658103 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096452 MemoryCapacity:32812007424 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f7568829fbf8dde6767b4a670268b SystemUUID:ec2f7568-829f-bf8d-de67-67b4a670268b BootID:1462693b-d101-453c-a329-02046e5e5229 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:75:64:c2:22:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:75:64:c2:22:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:d1:ea:c3:c1:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812007424 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:53:48.721616 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.721610    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:53:48.721783 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.721766    2576 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:53:48.724863 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.724835    2576 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:53:48.725028 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.724866    2576 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-166.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:53:48.725106 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.725032    2576 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:53:48.725106 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.725046    2576 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:53:48.725194 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.725116    2576 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:53:48.725194 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.725141    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:53:48.725924 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.725912    2576 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:53:48.729469 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.729455    2576 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:53:48.729603 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.729592    2576 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:53:48.730363 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.730352    2576 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:53:48.730442 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.730370    2576 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:53:48.730442 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.730386    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:53:48.730442 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.730418    2576 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:53:48.730442 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.730432    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:53:48.730872 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.730856    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:53:48.733295 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.733268    2576 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:53:48.737776 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.737764    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:53:48.738608 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738596    2576 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:53:48.738648 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738626    2576 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:53:48.738838 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738822    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:53:48.738865 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738850    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:53:48.738895 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738867    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:53:48.738895 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738880    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:53:48.738895 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738890    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:53:48.738970 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.738896    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:53:48.739563 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.739554    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:53:48.739594 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.739564    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:53:48.739594 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.739572    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:53:48.739594 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.739578    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:53:48.739594 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.739588    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:53:48.739782 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.739596    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:53:48.740932 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.740922    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:53:48.740969 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.740934    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:53:48.744820 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.744808    2576 server.go:1263] "Started kubelet"
Oct 30 21:53:48.744973 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.744927    2576 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:53:48.745032 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.744971    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:53:48.745104 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.745091    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:53:48.745728 ip-10-0-138-166 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:53:48.749001 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.748921    2576 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:53:48.751110 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.751093    2576 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:53:48.754474 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:48.754452    2576 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:53:48.756885 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.756860    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:53:48.757355 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.757331    2576 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:53:48.757456 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:48.757433    2576 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-166.us-west-1.compute.internal\" not found"
Oct 30 21:53:48.757893 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.757877    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:53:48.758126 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.757898    2576 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:53:48.758209 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758130    2576 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:53:48.758209 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758187    2576 factory.go:55] Registering systemd factory
Oct 30 21:53:48.758209 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758203    2576 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:53:48.758209 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758204    2576 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:53:48.758385 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758215    2576 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:53:48.758440 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758431    2576 factory.go:153] Registering CRI-O factory
Oct 30 21:53:48.758467 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758445    2576 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:53:48.758503 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758492    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:53:48.758531 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758514    2576 factory.go:103] Registering Raw factory
Oct 30 21:53:48.758531 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.758529    2576 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:53:48.759644 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.759630    2576 manager.go:319] Starting recovery of all containers
Oct 30 21:53:48.766313 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.766284    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:53:48.771699 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.771682    2576 manager.go:324] Recovery completed
Oct 30 21:53:48.775635 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.775620    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.777150 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777131    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.777220 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777165    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.777220 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777179    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:48.777681 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777667    2576 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:53:48.777728 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777682    2576 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:53:48.777728 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777704    2576 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:53:48.777728 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.777711    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.778460 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.778383    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.778517 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.778474    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.778517 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.778488    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:48.779805 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.779790    2576 policy_none.go:50] "Start"
Oct 30 21:53:48.779893 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.779808    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:53:48.779893 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.779821    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:53:48.781788 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.781776    2576 policy_none.go:44] "Start"
Oct 30 21:53:48.819969 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.819935    2576 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:53:48.823936 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.823794    2576 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:53:48.824021 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:48.823956    2576 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:53:48.824021 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.823968    2576 server.go:87] "Starting device plugin registration server"
Oct 30 21:53:48.824224 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.824211    2576 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:53:48.824283 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.824225    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:53:48.824361 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.824331    2576 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:53:48.824484 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.824467    2576 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:53:48.824589 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.824477    2576 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:53:48.824900 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:48.824883    2576 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:53:48.824962 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:48.824948    2576 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-166.us-west-1.compute.internal\" not found"
Oct 30 21:53:48.892163 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.892144    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:53:48.893998 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.892169    2576 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:53:48.893998 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.892184    2576 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:53:48.893998 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:48.892217    2576 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:53:48.925080 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.925031    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.925807 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.925792    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.925881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.925820    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.925881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.925834    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:48.925881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.925871    2576 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:48.971619 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.971596    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:53:48.971701 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.971652    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-frdbk"
Oct 30 21:53:48.982679 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.982664    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-frdbk"
Oct 30 21:53:48.992345 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.992324    2576 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"]
Oct 30 21:53:48.992422 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.992380    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.993218 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.993205    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.993285 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.993231    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.993285 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.993241    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:48.995467 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.995455    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.995620 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.995606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:48.995691 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.995660    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.996165 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.996149    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.996245 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.996155    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.996245 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.996195    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.996245 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.996213    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:48.996245 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.996172    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.996245 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.996243    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:48.998410 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.998380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:48.998508 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.998442    2576 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:48.999104 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.999086    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:48.999178 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.999120    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:48.999178 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:48.999135    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:53:49.025788 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:49.025772    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-166.us-west-1.compute.internal\" not found" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.028626 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:49.028611    2576 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-166.us-west-1.compute.internal\" not found" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.054090 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.054074    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:49.054688 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.054674    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:49.059965 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.059945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.060037 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.059974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.060037 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.059992    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c91ce82603c0eba2cf2b8be390abf9d9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"c91ce82603c0eba2cf2b8be390abf9d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.152809 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.152779    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:49.154611 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.154588    2576 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.154740 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.154613    2576 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:49.154740 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:49.154616    2576 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-166.us-west-1.compute.internal\": node \"ip-10-0-138-166.us-west-1.compute.internal\" not found"
Oct 30 21:53:49.158447 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.158431    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.160640 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.160625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.160683 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.160648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c91ce82603c0eba2cf2b8be390abf9d9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"c91ce82603c0eba2cf2b8be390abf9d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.160683 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.160674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.160746 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.160727    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.160795 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.160784    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c91ce82603c0eba2cf2b8be390abf9d9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"c91ce82603c0eba2cf2b8be390abf9d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.160828 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.160814    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.225099 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.225041    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:49.225191 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.225135    2576 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.283130 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.283106    2576 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:49.328825 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.328801    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.331797 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.331780    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:53:49.686799 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.686731    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:53:49.687271 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.686896    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:53:49.687271 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.686904    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:53:49.687271 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.686932    2576 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:53:49.687271 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:49.686958    2576 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a2a55f6a5a807453fa6f77ca235301fa-4f791996f38f588f.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-138-166.us-west-1.compute.internal.18e36d471a6e8eac\": read tcp 10.0.138.166:33244->18.116.183.115:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-138-166.us-west-1.compute.internal.18e36d471a6e8eac  default   5138 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-166.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-166.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-166.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:53:48 +0000 UTC,LastTimestamp:2026-10-30 21:53:48.925807129 +0000 UTC m=+0.518267294,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-166.us-west-1.compute.internal,}"
Oct 30 21:53:49.730834 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.730804    2576 apiserver.go:52] "Watching apiserver"
Oct 30 21:53:49.757635 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.757616    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:53:49.986997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.986920    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:48:49 +0000 UTC" deadline="2028-05-06 14:47:02.352768127 +0000 UTC"
Oct 30 21:53:49.986997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:49.986947    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13288h53m12.365823514s"
Oct 30 21:53:50.002454 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.002431    2576 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:53:50.002742 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.002726    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-g96mv","kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-gd8pt","openshift-network-operator/iptables-alerter-fq4cx","openshift-ovn-kubernetes/ovnkube-node-pk67r","kube-system/global-pull-secret-syncer-2xm8j","openshift-cluster-node-tuning-operator/tuned-86pjd","openshift-dns/node-resolver-2r6hv","openshift-multus/multus-fpbqf","openshift-multus/network-metrics-daemon-k4dcq","kube-system/konnectivity-agent-pjnv8","openshift-image-registry/node-ca-7x2zz"]
Oct 30 21:53:50.005142 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.005125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:50.005226 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.005205    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:53:50.007236 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.007223    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.009320 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.009301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.011460 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.011442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.013619 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.013602    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.015626 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.015610    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.015712 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.015674    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:53:50.017722 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.017704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.019867 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.019846    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.022058 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.022043    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.024452 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.024437    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.024541 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.024483    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:53:50.027442 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.027424    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.029625 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.029609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.059508 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.059489    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:53:50.066046 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-k8s-cni-cncf-io\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.066296 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-script-lib\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.066296 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.066296 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-device-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.066296 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-host-slash\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.066296 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysconfig\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.066498 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-lib-modules\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.066498 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjd6n\" (UniqueName: \"kubernetes.io/projected/834c4daa-eb50-46b5-919f-c201d9458e11-kube-api-access-cjd6n\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.066906 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.067020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067000    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:53:50.067094 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.066926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-systemd-units\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.067094 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-config\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.067191 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-sys-fs\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.067191 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-conf\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.067191 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-multus\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.067320 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067258    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.067450 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067430    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:53:50.067527 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067470    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.067575 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067548    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.067622 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067598    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.067746 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067729    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z7lww\""
Oct 30 21:53:50.067859 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067841    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:53:50.067906 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-etc-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.067939 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067922    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ghldd\""
Oct 30 21:53:50.067939 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.068009 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-netd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.068009 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.067985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-registration-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.068099 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068032    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:53:50.068099 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068031    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-socket-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.068099 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068085    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-host\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.068232 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068195    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:53:50.068232 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmk4b\" (UniqueName: \"kubernetes.io/projected/cebe5fd4-d933-4a1b-a147-e7c013444b0d-kube-api-access-cmk4b\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.068340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.068412 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-modprobe-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.068488 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.068543 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-netns\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.068598 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068580    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cnibin\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.068653 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.068703 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.068703 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-slash\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.068803 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-kubelet-config\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.068860 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068788    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-tuned\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.068860 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cebe5fd4-d933-4a1b-a147-e7c013444b0d-tmp-dir\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.068955 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-bin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.068955 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/834c4daa-eb50-46b5-919f-c201d9458e11-serviceca\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.068955 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.069096 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-log-socket\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.069096 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.068999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46124db8-6b86-4a74-8ef7-80a2a04a8463-agent-certs\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.069096 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbn98\" (UniqueName: \"kubernetes.io/projected/94ac4f3c-9331-4223-bad5-dd4299321eb6-kube-api-access-fbn98\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.069295 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cebe5fd4-d933-4a1b-a147-e7c013444b0d-hosts-file\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.069381 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069334    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-hostroot\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.069455 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-conf-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.069489 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069462    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-var-lib-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.069574 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.069877 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-system-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.069908 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tljcv\" (UniqueName: \"kubernetes.io/projected/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kube-api-access-tljcv\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.069942 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-systemd\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.069973 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-run\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.070011 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.069996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-cnibin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070042 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-socket-dir-parent\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070078 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:50.070118 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-system-cni-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.070118 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070201 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-env-overrides\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070201 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-systemd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070278 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070218    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-var-lib-kubelet\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.070278 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-tmp\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.070363 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-multus-certs\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070363 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-etc-kubernetes\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070465 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sdz9\" (UniqueName: \"kubernetes.io/projected/06a83215-f446-480d-9036-69d4ac88e172-kube-api-access-8sdz9\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070465 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-netns\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070465 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovn-node-metrics-cert\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070465 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-dbus\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.070591 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-kubelet\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070591 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.070591 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-kubelet\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070591 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-ovn\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070591 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070572    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89lw7\" (UniqueName: \"kubernetes.io/projected/2be95b15-43a7-4f7e-85c7-14a0adccb875-kube-api-access-89lw7\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070724 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76xfx\" (UniqueName: \"kubernetes.io/projected/b801f58e-2e25-4e89-b099-030d98aa44c9-kube-api-access-76xfx\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.070724 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.070724 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-cni-binary-copy\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070724 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-kubernetes\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.070724 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-multus-daemon-config\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.070724 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-iptables-alerter-script\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-kube-api-access-zwkrc\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070758    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834c4daa-eb50-46b5-919f-c201d9458e11-host\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-node-log\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-bin\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-sys\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rm6\" (UniqueName: \"kubernetes.io/projected/29c88d96-2e8f-4431-9975-f4e8ecba95ba-kube-api-access-j2rm6\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.070881 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.071086 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.070888    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46124db8-6b86-4a74-8ef7-80a2a04a8463-konnectivity-ca\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.073483 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.073469    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.169473 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.169451    2576 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:53:50.169589 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.169531    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvb7x"
Oct 30 21:53:50.170582 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170516    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:53:50.170582 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170530    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:53:50.170582 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170537    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:53:50.170582 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170545    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t4xfw\""
Oct 30 21:53:50.170796 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170688    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:53:50.170796 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170733    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.170858 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170796    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.170858 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170836    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pgbrx\""
Oct 30 21:53:50.171075 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.170992    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.171075 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171017    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vjl7c\""
Oct 30 21:53:50.171075 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171045    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hbp5d\""
Oct 30 21:53:50.171075 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.171075 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171069    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-modprobe-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171100    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8jkh4\""
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171111    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-netns\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171111    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s9wtv\""
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cnibin\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.171171    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171192    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-netns\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171216    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171216    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-modprobe-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171223    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cnibin\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.171254    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret podName:d64e1a7d-50f4-4f15-ab5c-71921ae2af7e nodeName:}" failed. No retries permitted until 2026-10-30 21:53:50.671215081 +0000 UTC m=+2.263675247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret") pod "global-pull-secret-syncer-2xm8j" (UID: "d64e1a7d-50f4-4f15-ab5c-71921ae2af7e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171308    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:53:50.171340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-slash\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171356    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-kubelet-config\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-tuned\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171373    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cebe5fd4-d933-4a1b-a147-e7c013444b0d-tmp-dir\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-bin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171449    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-slash\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171453    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-kubelet-config\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/834c4daa-eb50-46b5-919f-c201d9458e11-serviceca\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171507    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-bin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171526    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-log-socket\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46124db8-6b86-4a74-8ef7-80a2a04a8463-agent-certs\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbn98\" (UniqueName: \"kubernetes.io/projected/94ac4f3c-9331-4223-bad5-dd4299321eb6-kube-api-access-fbn98\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171601    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cebe5fd4-d933-4a1b-a147-e7c013444b0d-hosts-file\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171623    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-hostroot\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-conf-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171703    2576 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:53:50.172113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171762    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cebe5fd4-d933-4a1b-a147-e7c013444b0d-tmp-dir\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171808    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171849    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-hostroot\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-var-lib-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171877    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/834c4daa-eb50-46b5-919f-c201d9458e11-serviceca\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171907    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cebe5fd4-d933-4a1b-a147-e7c013444b0d-hosts-file\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171916    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-log-socket\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-system-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171934    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171949    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tljcv\" (UniqueName: \"kubernetes.io/projected/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kube-api-access-tljcv\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171965    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-var-lib-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171959    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-conf-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.171982    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-system-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-systemd\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-run\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-cnibin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173006 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172118    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-systemd\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-socket-dir-parent\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-system-cni-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172196    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-cnibin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-env-overrides\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-systemd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172245    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-run\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172254    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-system-cni-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-var-lib-kubelet\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172282    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-tmp\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-multus-certs\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172318    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-socket-dir-parent\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172321    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-systemd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-etc-kubernetes\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.173844 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172358    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-var-lib-kubelet\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdz9\" (UniqueName: \"kubernetes.io/projected/06a83215-f446-480d-9036-69d4ac88e172-kube-api-access-8sdz9\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172411    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-etc-kubernetes\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-netns\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172459    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-multus-certs\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172487    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-netns\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovn-node-metrics-cert\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-dbus\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-kubelet\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-kubelet\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-ovn\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89lw7\" (UniqueName: \"kubernetes.io/projected/2be95b15-43a7-4f7e-85c7-14a0adccb875-kube-api-access-89lw7\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xfx\" (UniqueName: \"kubernetes.io/projected/b801f58e-2e25-4e89-b099-030d98aa44c9-kube-api-access-76xfx\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172685    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-dbus\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172709    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-kubelet\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.174451 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-cni-binary-copy\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172744    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-kubelet\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172757    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-kubernetes\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-multus-daemon-config\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172808    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-iptables-alerter-script\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-kube-api-access-zwkrc\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834c4daa-eb50-46b5-919f-c201d9458e11-host\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-node-log\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-bin\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-sys\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172946    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-ovn\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rm6\" (UniqueName: \"kubernetes.io/projected/29c88d96-2e8f-4431-9975-f4e8ecba95ba-kube-api-access-j2rm6\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46124db8-6b86-4a74-8ef7-80a2a04a8463-konnectivity-ca\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-k8s-cni-cncf-io\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-script-lib\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.175210 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-device-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-host-slash\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysconfig\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-lib-modules\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjd6n\" (UniqueName: \"kubernetes.io/projected/834c4daa-eb50-46b5-919f-c201d9458e11-kube-api-access-cjd6n\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-systemd-units\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-config\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173348    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-sys-fs\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-conf\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173385    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.172682    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-env-overrides\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-multus\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173448    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysconfig\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-etc-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173469    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-multus\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.175773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-netd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-registration-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173562    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-lib-modules\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-socket-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173700    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-socket-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173785    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173791    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173837    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-systemd-units\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173293    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173935    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46124db8-6b86-4a74-8ef7-80a2a04a8463-konnectivity-ca\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.173996    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-k8s-cni-cncf-io\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174225    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-kubernetes\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174364    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834c4daa-eb50-46b5-919f-c201d9458e11-host\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174617    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-device-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174681    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-iptables-alerter-script\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174726    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-conf\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174749    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-node-log\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176535 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-host\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174784    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-sys-fs\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmk4b\" (UniqueName: \"kubernetes.io/projected/cebe5fd4-d933-4a1b-a147-e7c013444b0d-kube-api-access-cmk4b\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174831    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-host-slash\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174832    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-cni-binary-copy\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174888    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-netd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174908    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-sys\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.174949    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-etc-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.175002    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-registration-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.175028    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.175033    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-host\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.175128    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.175177    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:50.675161428 +0000 UTC m=+2.267621597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.175128    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-bin\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.175723    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-tmp\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.176043    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46124db8-6b86-4a74-8ef7-80a2a04a8463-agent-certs\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.176576    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-tuned\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.176997 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.176624    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovn-node-metrics-cert\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.220511 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.220487    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.220511 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.220487    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.221552 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.221530    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:53:50.221552 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.221541    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.221552 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.221551    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:53:50.221803 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.221583    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:53:50.221803 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.221606    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4dnxg\""
Oct 30 21:53:50.221803 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.221605    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:53:50.222071 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.222053    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:53:50.225763 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.225747    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-multus-daemon-config\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.225811 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.225772    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-script-lib\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.226074 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.226059    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-config\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.233170 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.233153    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:50.233266 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.233176    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:50.233266 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.233190    2576 projected.go:196] Error preparing data for projected volume kube-api-access-95k2j for pod openshift-network-diagnostics/network-check-target-g96mv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:50.233266 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.233256    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j podName:eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:50.733237934 +0000 UTC m=+2.325698104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-95k2j" (UniqueName: "kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j") pod "network-check-target-g96mv" (UID: "eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:50.235310 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.235287    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbn98\" (UniqueName: \"kubernetes.io/projected/94ac4f3c-9331-4223-bad5-dd4299321eb6-kube-api-access-fbn98\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.235419 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.235382    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sdz9\" (UniqueName: \"kubernetes.io/projected/06a83215-f446-480d-9036-69d4ac88e172-kube-api-access-8sdz9\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.235878 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.235852    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tljcv\" (UniqueName: \"kubernetes.io/projected/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kube-api-access-tljcv\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.235953 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.235913    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xfx\" (UniqueName: \"kubernetes.io/projected/b801f58e-2e25-4e89-b099-030d98aa44c9-kube-api-access-76xfx\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.240628 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.240581    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjd6n\" (UniqueName: \"kubernetes.io/projected/834c4daa-eb50-46b5-919f-c201d9458e11-kube-api-access-cjd6n\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.240753 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.240734    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-kube-api-access-zwkrc\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.240753 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.240747    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89lw7\" (UniqueName: \"kubernetes.io/projected/2be95b15-43a7-4f7e-85c7-14a0adccb875-kube-api-access-89lw7\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.242030 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.242015    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmk4b\" (UniqueName: \"kubernetes.io/projected/cebe5fd4-d933-4a1b-a147-e7c013444b0d-kube-api-access-cmk4b\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.242745 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.242729    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rm6\" (UniqueName: \"kubernetes.io/projected/29c88d96-2e8f-4431-9975-f4e8ecba95ba-kube-api-access-j2rm6\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.357325 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.357290    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:53:50.381866 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.381841    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:53:50.383993 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.383970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:53:50.401722 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.401704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:53:50.405861 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.405842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:53:50.429857 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.429837    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:53:50.435696 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.435675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fpbqf"
Oct 30 21:53:50.446003 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.445975    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ac4f3c_9331_4223_bad5_dd4299321eb6.slice/crio-24b051f12f23c2ed82363bd531f1b412260a46f2cef73e51028926e27cacc522 WatchSource:0}: Error finding container 24b051f12f23c2ed82363bd531f1b412260a46f2cef73e51028926e27cacc522: Status 404 returned error can't find the container with id 24b051f12f23c2ed82363bd531f1b412260a46f2cef73e51028926e27cacc522
Oct 30 21:53:50.446608 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.446584    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91ce82603c0eba2cf2b8be390abf9d9.slice/crio-e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd WatchSource:0}: Error finding container e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd: Status 404 returned error can't find the container with id e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd
Oct 30 21:53:50.447247 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.447227    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcebe5fd4_d933_4a1b_a147_e7c013444b0d.slice/crio-b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125 WatchSource:0}: Error finding container b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125: Status 404 returned error can't find the container with id b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125
Oct 30 21:53:50.452532 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.452517    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:53:50.458697 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.458675    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a83215_f446_480d_9036_69d4ac88e172.slice/crio-044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e WatchSource:0}: Error finding container 044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e: Status 404 returned error can't find the container with id 044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e
Oct 30 21:53:50.459241 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.459220    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f411b386155968c36dc88fdca50807e.slice/crio-d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232 WatchSource:0}: Error finding container d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232: Status 404 returned error can't find the container with id d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232
Oct 30 21:53:50.460322 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.460303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:53:50.464999 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.464985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:53:50.469165 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.469147    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46124db8_6b86_4a74_8ef7_80a2a04a8463.slice/crio-0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6 WatchSource:0}: Error finding container 0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6: Status 404 returned error can't find the container with id 0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6
Oct 30 21:53:50.473515 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:50.473496    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834c4daa_eb50_46b5_919f_c201d9458e11.slice/crio-01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442 WatchSource:0}: Error finding container 01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442: Status 404 returned error can't find the container with id 01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442
Oct 30 21:53:50.678851 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.678815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:50.678987 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.678874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:50.678987 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.678965    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:50.678987 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.678986    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:50.679110 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.679039    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret podName:d64e1a7d-50f4-4f15-ab5c-71921ae2af7e nodeName:}" failed. No retries permitted until 2026-10-30 21:53:51.679014162 +0000 UTC m=+3.271474331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret") pod "global-pull-secret-syncer-2xm8j" (UID: "d64e1a7d-50f4-4f15-ab5c-71921ae2af7e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:50.679110 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.679062    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:51.679051043 +0000 UTC m=+3.271511197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:50.686610 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.686583    2576 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:50.779469 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.779424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:50.780109 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.779577    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:50.780109 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.779597    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:50.780109 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.779607    2576 projected.go:196] Error preparing data for projected volume kube-api-access-95k2j for pod openshift-network-diagnostics/network-check-target-g96mv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:50.780109 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:50.779660    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j podName:eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:51.77964677 +0000 UTC m=+3.372106939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-95k2j" (UniqueName: "kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j") pod "network-check-target-g96mv" (UID: "eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:50.877315 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.877281    2576 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:50.886639 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.886610    2576 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:50.896812 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.896730    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"3f411b386155968c36dc88fdca50807e","Type":"ContainerStarted","Data":"d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232"}
Oct 30 21:53:50.897920 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.897891    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fpbqf" event={"ID":"06a83215-f446-480d-9036-69d4ac88e172","Type":"ContainerStarted","Data":"044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e"}
Oct 30 21:53:50.899030 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.899005    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2r6hv" event={"ID":"cebe5fd4-d933-4a1b-a147-e7c013444b0d","Type":"ContainerStarted","Data":"b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125"}
Oct 30 21:53:50.900057 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.900033    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"c91ce82603c0eba2cf2b8be390abf9d9","Type":"ContainerStarted","Data":"e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd"}
Oct 30 21:53:50.901097 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.901077    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-86pjd" event={"ID":"94ac4f3c-9331-4223-bad5-dd4299321eb6","Type":"ContainerStarted","Data":"24b051f12f23c2ed82363bd531f1b412260a46f2cef73e51028926e27cacc522"}
Oct 30 21:53:50.902151 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.902133    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7x2zz" event={"ID":"834c4daa-eb50-46b5-919f-c201d9458e11","Type":"ContainerStarted","Data":"01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442"}
Oct 30 21:53:50.905583 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:50.905560    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjnv8" event={"ID":"46124db8-6b86-4a74-8ef7-80a2a04a8463","Type":"ContainerStarted","Data":"0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6"}
Oct 30 21:53:51.173339 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.173235    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:48:50 +0000 UTC" deadline="2028-07-28 13:51:58.198454442 +0000 UTC"
Oct 30 21:53:51.173339 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.173277    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15279h58m7.025180162s"
Oct 30 21:53:51.237372 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:51.237324    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c062cd_a506_47bf_ad60_4bcfe3c5a19e.slice/crio-177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41 WatchSource:0}: Error finding container 177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41: Status 404 returned error can't find the container with id 177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41
Oct 30 21:53:51.273243 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:51.273217    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c88d96_2e8f_4431_9975_f4e8ecba95ba.slice/crio-77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b WatchSource:0}: Error finding container 77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b: Status 404 returned error can't find the container with id 77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b
Oct 30 21:53:51.356604 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:53:51.356575    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be95b15_43a7_4f7e_85c7_14a0adccb875.slice/crio-33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927 WatchSource:0}: Error finding container 33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927: Status 404 returned error can't find the container with id 33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927
Oct 30 21:53:51.686915 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.686477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:51.686915 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.686545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:51.686915 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.686710    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:51.686915 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.686778    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret podName:d64e1a7d-50f4-4f15-ab5c-71921ae2af7e nodeName:}" failed. No retries permitted until 2026-10-30 21:53:53.686758672 +0000 UTC m=+5.279218829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret") pod "global-pull-secret-syncer-2xm8j" (UID: "d64e1a7d-50f4-4f15-ab5c-71921ae2af7e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:51.686915 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.686848    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:51.686915 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.686882    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:53.68687079 +0000 UTC m=+5.279330948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:51.788129 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.787790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:51.788129 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.787988    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:51.788129 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.788011    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:51.788129 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.788025    2576 projected.go:196] Error preparing data for projected volume kube-api-access-95k2j for pod openshift-network-diagnostics/network-check-target-g96mv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:51.788129 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.788086    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j podName:eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:53.788066756 +0000 UTC m=+5.380526929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-95k2j" (UniqueName: "kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j") pod "network-check-target-g96mv" (UID: "eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:51.892656 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.892619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:51.892849 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.892782    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:53:51.892926 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.892871    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:51.892983 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.892946    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:53:51.893041 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.893028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:51.893118 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:51.893098    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:53:51.916880 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.916838    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927"}
Oct 30 21:53:51.923005 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.922965    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerStarted","Data":"77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b"}
Oct 30 21:53:51.929131 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.929099    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fq4cx" event={"ID":"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95","Type":"ContainerStarted","Data":"12c3c632cb2610779d1d7078a0da4366d394e1a9df4d3d0624c84f3d2daddd72"}
Oct 30 21:53:51.931490 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:51.931450    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9" event={"ID":"80c062cd-a506-47bf-ad60-4bcfe3c5a19e","Type":"ContainerStarted","Data":"177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41"}
Oct 30 21:53:52.174349 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:52.174208    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:48:50 +0000 UTC" deadline="2028-05-13 13:35:48.669158707 +0000 UTC"
Oct 30 21:53:52.174349 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:52.174299    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13455h41m56.494864575s"
Oct 30 21:53:53.706869 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:53.706160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:53.706869 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:53.706276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:53.706869 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.706435    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:53.706869 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.706504    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:57.706484466 +0000 UTC m=+9.298944637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:53.706869 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.706567    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:53.706869 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.706646    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret podName:d64e1a7d-50f4-4f15-ab5c-71921ae2af7e nodeName:}" failed. No retries permitted until 2026-10-30 21:53:57.706617613 +0000 UTC m=+9.299077780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret") pod "global-pull-secret-syncer-2xm8j" (UID: "d64e1a7d-50f4-4f15-ab5c-71921ae2af7e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:53.807471 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:53.806974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:53.807471 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.807314    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:53.807471 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.807337    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:53.807471 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.807350    2576 projected.go:196] Error preparing data for projected volume kube-api-access-95k2j for pod openshift-network-diagnostics/network-check-target-g96mv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:53.807471 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.807429    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j podName:eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:57.807410354 +0000 UTC m=+9.399870527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-95k2j" (UniqueName: "kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j") pod "network-check-target-g96mv" (UID: "eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:53.892971 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:53.892864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:53.893140 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.892977    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:53:53.893140 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:53.893046    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:53.893140 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.893125    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:53:53.893313 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:53.893184    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:53.893313 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:53.893259    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:53:55.893304 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.893002    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:55.893794 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:55.893363    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:53:55.893863 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.893802    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:55.893918 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:55.893901    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:53:55.894031 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.893975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:55.894095 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:55.894044    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:53:55.949468 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.948705    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjnv8" event={"ID":"46124db8-6b86-4a74-8ef7-80a2a04a8463","Type":"ContainerStarted","Data":"9d102c402c5f33c35ca90a740cbb697b66217a9e15236e60440fc3ef4a9184ca"}
Oct 30 21:53:55.952920 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.952477    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"c91ce82603c0eba2cf2b8be390abf9d9","Type":"ContainerStarted","Data":"4ac5084a3009b08ecda012640c5e8032e051c56a645275706882f845d57e6e74"}
Oct 30 21:53:55.958910 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.958252    2576 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="978cc22ff4f9d153b69e5eb4ec49f7444b9e34326879095cb90d023c8bf3df75" exitCode=0
Oct 30 21:53:55.958910 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.958322    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerDied","Data":"978cc22ff4f9d153b69e5eb4ec49f7444b9e34326879095cb90d023c8bf3df75"}
Oct 30 21:53:55.963433 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.962882    2576 generic.go:356] "Generic (PLEG): container finished" podID="3f411b386155968c36dc88fdca50807e" containerID="352330a7f52e422b574b5551fe6062c83c43b1b93feab10e7675630e3ebd3bdb" exitCode=0
Oct 30 21:53:55.963433 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:55.962919    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"3f411b386155968c36dc88fdca50807e","Type":"ContainerDied","Data":"352330a7f52e422b574b5551fe6062c83c43b1b93feab10e7675630e3ebd3bdb"}
Oct 30 21:53:56.069864 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:56.069644    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pjnv8" podStartSLOduration=2.235528613 podStartE2EDuration="7.069630706s" totalImagesPullingTime="4.834102093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:56.068542156 +0000 UTC m=+7.661002332" watchObservedRunningTime="2026-10-30 21:53:56.069630706 +0000 UTC m=+7.662090880"
Oct 30 21:53:56.973959 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:56.973921    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"3f411b386155968c36dc88fdca50807e","Type":"ContainerStarted","Data":"7e30097d147c7e7c12d542c3c30dc4ff592776576e6598de9c1aa7f519f14d0d"}
Oct 30 21:53:56.979767 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:56.979743    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7x2zz" event={"ID":"834c4daa-eb50-46b5-919f-c201d9458e11","Type":"ContainerStarted","Data":"412b3c25dbb4bfee45879db0bd4e34d82d8889cc40676768df61b526af145a92"}
Oct 30 21:53:57.097633 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.097570    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal" podStartSLOduration=8.097551014 podStartE2EDuration="8.097551014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:56.457538586 +0000 UTC m=+8.049998760" watchObservedRunningTime="2026-10-30 21:53:57.097551014 +0000 UTC m=+8.690011191"
Oct 30 21:53:57.097835 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.097757    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" podStartSLOduration=8.097749864 podStartE2EDuration="8.097749864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:57.09703876 +0000 UTC m=+8.689498937" watchObservedRunningTime="2026-10-30 21:53:57.097749864 +0000 UTC m=+8.690210040"
Oct 30 21:53:57.739898 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.739819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:57.739898 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.739879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:57.740106 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.740007    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:57.740106 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.740071    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret podName:d64e1a7d-50f4-4f15-ab5c-71921ae2af7e nodeName:}" failed. No retries permitted until 2026-10-30 21:54:05.74005323 +0000 UTC m=+17.332513400 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret") pod "global-pull-secret-syncer-2xm8j" (UID: "d64e1a7d-50f4-4f15-ab5c-71921ae2af7e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:57.740629 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.740604    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:57.740730 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.740673    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:05.740654288 +0000 UTC m=+17.333114457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:57.841047 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.841011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:57.841362 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.841340    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:57.841546 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.841368    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:57.841546 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.841381    2576 projected.go:196] Error preparing data for projected volume kube-api-access-95k2j for pod openshift-network-diagnostics/network-check-target-g96mv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:57.841546 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.841461    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j podName:eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:05.841441152 +0000 UTC m=+17.433901318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-95k2j" (UniqueName: "kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j") pod "network-check-target-g96mv" (UID: "eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:57.893063 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.892695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:57.893063 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.892807    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:53:57.893063 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.892873    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:57.893063 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.892925    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:53:57.893063 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:57.892966    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:57.893063 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:57.893019    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:53:58.989458 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:58.988489    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fpbqf" event={"ID":"06a83215-f446-480d-9036-69d4ac88e172","Type":"ContainerStarted","Data":"c09676c91e0e15200686a693e413898a40f4d93544bf32d1abd4514f31ecd114"}
Oct 30 21:53:58.996144 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:58.996108    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9" event={"ID":"80c062cd-a506-47bf-ad60-4bcfe3c5a19e","Type":"ContainerStarted","Data":"4fcdf1d51980e61979c8241c7f0d661909b76b647c6d0a5371805609475dc697"}
Oct 30 21:53:59.117831 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:59.117731    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7x2zz" podStartSLOduration=4.300572602 podStartE2EDuration="10.117711721s" totalImagesPullingTime="5.817139119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:57.218787265 +0000 UTC m=+8.811247439" watchObservedRunningTime="2026-10-30 21:53:59.117711721 +0000 UTC m=+10.710171896"
Oct 30 21:53:59.893555 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:59.893527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:53:59.893691 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:59.893664    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:53:59.893812 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:59.893764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:53:59.893962 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:59.893931    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:53:59.894045 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:59.893978    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:53:59.894100 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:53:59.894054    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:53:59.911956 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:53:59.911914    2576 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:54:00.000922 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.000884    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9" event={"ID":"80c062cd-a506-47bf-ad60-4bcfe3c5a19e","Type":"ContainerStarted","Data":"2284c2ceda91f6b343545eeca0df3684384f673a459967539e665ba006d02b7d"}
Oct 30 21:54:00.003475 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.003448    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2r6hv" event={"ID":"cebe5fd4-d933-4a1b-a147-e7c013444b0d","Type":"ContainerStarted","Data":"9f5398000a40bbeedba159cfab349fd20fa22e08c6f6eef898ee61a38d6e62a1"}
Oct 30 21:54:00.028799 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.028675    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:00.029293 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.029271    2576 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:00.125428 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.124088    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-fpbqf" podStartSLOduration=2.771038719 podStartE2EDuration="11.12406684s" totalImagesPullingTime="8.353028121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:59.118978489 +0000 UTC m=+10.711438665" watchObservedRunningTime="2026-10-30 21:54:00.12406684 +0000 UTC m=+11.716527013"
Oct 30 21:54:00.253253 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.253199    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-2r6hv" podStartSLOduration=2.317551934 podStartE2EDuration="11.25318417s" totalImagesPullingTime="8.935632236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:00.125210055 +0000 UTC m=+11.717670230" watchObservedRunningTime="2026-10-30 21:54:00.25318417 +0000 UTC m=+11.845644372"
Oct 30 21:54:00.833195 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.832804    2576 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:53:59.911930476Z","UUID":"1a1e2a6a-74ff-42c0-876c-81c73c69d068","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:54:00.834791 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.834753    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:54:00.834791 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:00.834788    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:54:01.008516 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:01.007910    2576 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="7b09d42c1320dda70f761e9cef2b873c5111d8eb38bf478461c055ccdd33ee19" exitCode=0
Oct 30 21:54:01.008516 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:01.008098    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerDied","Data":"7b09d42c1320dda70f761e9cef2b873c5111d8eb38bf478461c055ccdd33ee19"}
Oct 30 21:54:01.892922 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:01.892888    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:01.893079 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:01.893020    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:01.893628 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:01.893388    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:01.893628 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:01.893498    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:01.893628 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:01.893561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:01.893803 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:01.893636    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:02.012620 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:02.012583    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9" event={"ID":"80c062cd-a506-47bf-ad60-4bcfe3c5a19e","Type":"ContainerStarted","Data":"544d83c1959dcd9d123c8cb352a3e8096804ee8d79b29787c55d9e30480a25a9"}
Oct 30 21:54:02.014246 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:02.014219    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-86pjd" event={"ID":"94ac4f3c-9331-4223-bad5-dd4299321eb6","Type":"ContainerStarted","Data":"bf60cea6e248e3d2f9053bd288f053c977216cf505ac346bba95392333903c72"}
Oct 30 21:54:02.015800 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:02.015771    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fq4cx" event={"ID":"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95","Type":"ContainerStarted","Data":"bbacfdd6ff96417f840d452375ecd3d22584c391704844011be27f5be72cb9c3"}
Oct 30 21:54:02.137051 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:02.137006    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9" podStartSLOduration=3.370952408 podStartE2EDuration="13.136992765s" totalImagesPullingTime="9.766040357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:02.136850902 +0000 UTC m=+13.729311076" watchObservedRunningTime="2026-10-30 21:54:02.136992765 +0000 UTC m=+13.729452940"
Oct 30 21:54:02.259730 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:02.259672    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fq4cx" podStartSLOduration=4.951561905 podStartE2EDuration="13.259659989s" totalImagesPullingTime="8.308098084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:02.259546453 +0000 UTC m=+13.852006636" watchObservedRunningTime="2026-10-30 21:54:02.259659989 +0000 UTC m=+13.852120162"
Oct 30 21:54:02.387506 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:02.387453    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-86pjd" podStartSLOduration=2.450025642 podStartE2EDuration="13.387437951s" totalImagesPullingTime="10.937412309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:02.386659319 +0000 UTC m=+13.979119497" watchObservedRunningTime="2026-10-30 21:54:02.387437951 +0000 UTC m=+13.979898124"
Oct 30 21:54:03.020858 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:03.020818    2576 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="783cdb154bcf555fa439b57843d139d0071d55f67d7d61fa2d197c41284c59ab" exitCode=0
Oct 30 21:54:03.021532 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:03.020929    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerDied","Data":"783cdb154bcf555fa439b57843d139d0071d55f67d7d61fa2d197c41284c59ab"}
Oct 30 21:54:03.893310 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:03.893272    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:03.893504 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:03.893416    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:03.893504 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:03.893452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:03.893620 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:03.893568    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:03.893620 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:03.893599    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:03.893706 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:03.893670    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:05.028145 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.027921    2576 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="973dd411af0169002ab3952f7d8de082d0cb4547727c2f663d0b48dd2dabf48a" exitCode=0
Oct 30 21:54:05.028145 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.027981    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerDied","Data":"973dd411af0169002ab3952f7d8de082d0cb4547727c2f663d0b48dd2dabf48a"}
Oct 30 21:54:05.809323 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.809293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:05.809512 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.809352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:05.809512 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.809453    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:54:05.809512 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.809470    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:54:05.809512 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.809502    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:21.80948968 +0000 UTC m=+33.401949832 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:54:05.809708 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.809528    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret podName:d64e1a7d-50f4-4f15-ab5c-71921ae2af7e nodeName:}" failed. No retries permitted until 2026-10-30 21:54:21.809510874 +0000 UTC m=+33.401971032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret") pod "global-pull-secret-syncer-2xm8j" (UID: "d64e1a7d-50f4-4f15-ab5c-71921ae2af7e") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:54:05.892783 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.892750    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:05.892783 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.892774    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:05.893011 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.892886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:05.893011 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.892898    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:05.893115 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.892999    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:05.893115 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.893094    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:05.909977 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:05.909948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:05.910128 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.910097    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:54:05.910128 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.910114    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:54:05.910128 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.910127    2576 projected.go:196] Error preparing data for projected volume kube-api-access-95k2j for pod openshift-network-diagnostics/network-check-target-g96mv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:54:05.910240 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:05.910178    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j podName:eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:21.910161681 +0000 UTC m=+33.502621834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-95k2j" (UniqueName: "kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j") pod "network-check-target-g96mv" (UID: "eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:54:07.038359 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.038173    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"de1f0234b2aa5010928bded59b657832e28342aa757993e7540a83687c97daec"}
Oct 30 21:54:07.038694 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.038470    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"5aa7077b271e240f760d9822c1c04f3099c0f6a4c41cc313e0a9dad721156076"}
Oct 30 21:54:07.038694 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.038530    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"fc89207c4907c579364b5203bb33c2bc1c7faaa7be52f3b46bd70e3d7c55153e"}
Oct 30 21:54:07.038694 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.038562    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"0c941fc8418c93577a2a3cf7f72b5983d6deb549772494be379d0815c9a1a1bf"}
Oct 30 21:54:07.038694 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.038609    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"486501eb7ccde0fba4d3127edfe0874eb3fe021e97217a23ebeff44396eb05c9"}
Oct 30 21:54:07.038694 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.038678    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"ed25f7751ceca03900a777161cdf62e4e962907719e02316c1954c57cf79c00e"}
Oct 30 21:54:07.892382 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.892345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:07.892557 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.892507    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:07.892557 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:07.892536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:07.892557 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:07.892522    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:07.892703 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:07.892604    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:07.892703 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:07.892656    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:08.043287 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:08.043252    2576 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="bc4b6e12db1c6ef185a7a25ec3581e0d0461a6096f09f55e4c2998c9fdf6900e" exitCode=0
Oct 30 21:54:08.043752 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:08.043304    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerDied","Data":"bc4b6e12db1c6ef185a7a25ec3581e0d0461a6096f09f55e4c2998c9fdf6900e"}
Oct 30 21:54:09.048604 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:09.048572    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"473067dd6fbb18ac9e1ed0553b72e67536beeb36b03527965e2492066a3290cd"}
Oct 30 21:54:09.051480 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:09.051454    2576 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="293c884dc490fd09f5c91b7a30988dd4dffc4f185c0d222e414052410152adf2" exitCode=0
Oct 30 21:54:09.051608 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:09.051498    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerDied","Data":"293c884dc490fd09f5c91b7a30988dd4dffc4f185c0d222e414052410152adf2"}
Oct 30 21:54:09.892922 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:09.892883    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:09.892922 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:09.892909    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:09.893119 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:09.893010    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:09.893119 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:09.893040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:09.893119 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:09.893099    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:09.893208 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:09.893141    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:10.056839 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:10.056799    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" event={"ID":"29c88d96-2e8f-4431-9975-f4e8ecba95ba","Type":"ContainerStarted","Data":"2f318ad36429f91f7bace74cf8b0471803e65323c4f9ecff906b2eb510baf959"}
Oct 30 21:54:10.196859 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:10.196757    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gd8pt" podStartSLOduration=9.426219279 podStartE2EDuration="21.196744404s" totalImagesPullingTime="11.770525125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:10.196166876 +0000 UTC m=+21.788627049" watchObservedRunningTime="2026-10-30 21:54:10.196744404 +0000 UTC m=+21.789204579"
Oct 30 21:54:11.063196 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.062970    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" event={"ID":"2be95b15-43a7-4f7e-85c7-14a0adccb875","Type":"ContainerStarted","Data":"288391ea196704ac8cad7f8182796c28220889dc94c5ae92cfa658aae70eafa2"}
Oct 30 21:54:11.063773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.063433    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:11.063773 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.063500    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:11.079504 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.079481    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:11.349531 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.349390    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r" podStartSLOduration=7.758570054 podStartE2EDuration="22.34937604s" totalImagesPullingTime="14.590805986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:11.348885207 +0000 UTC m=+22.941345383" watchObservedRunningTime="2026-10-30 21:54:11.34937604 +0000 UTC m=+22.941836213"
Oct 30 21:54:11.892808 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.892483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:11.892808 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.892483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:11.893062 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:11.892904    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:11.893062 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:11.892957    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:11.893130 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:11.893060    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:11.893162 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:11.893124    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:12.065355 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:12.065326    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:12.080735 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:12.080708    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:13.381115 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:13.381051    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2xm8j"]
Oct 30 21:54:13.381549 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:13.381149    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:13.381549 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:13.381229    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:13.384275 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:13.384228    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k4dcq"]
Oct 30 21:54:13.384446 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:13.384354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:13.384535 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:13.384492    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:13.384966 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:13.384945    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g96mv"]
Oct 30 21:54:13.385046 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:13.385035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:13.385134 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:13.385113    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:14.892391 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:14.892351    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:14.892391 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:14.892376    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:14.892933 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:14.892496    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:14.892933 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:14.892512    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:14.892933 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:14.892575    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:14.892933 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:14.892656    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:16.600368 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:16.600329    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:16.600878 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:16.600538    2576 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:54:16.601013 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:16.600986    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:16.895778 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:16.895697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:16.895925 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:16.895820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:16.895925 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:16.895833    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:16.895925 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:16.895850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:16.896092 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:16.895941    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:16.896092 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:16.896035    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:18.899126 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:18.898933    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:18.899568 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:18.899009    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:18.899568 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:18.899208    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2xm8j" podUID="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e"
Oct 30 21:54:18.899568 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:18.899017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:18.899568 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:18.899270    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4dcq" podUID="b801f58e-2e25-4e89-b099-030d98aa44c9"
Oct 30 21:54:18.899568 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:18.899336    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g96mv" podUID="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1"
Oct 30 21:54:19.709892 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.709868    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:54:19.710041 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.709985    2576 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:54:19.829932 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.829848    2576 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:54:19.870010 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.869973    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vx6lh"]
Oct 30 21:54:19.889827 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.889801    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6w2gg"]
Oct 30 21:54:19.889978 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.889939    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:19.899850 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.899831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:19.944055 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.944003    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vx6lh"]
Oct 30 21:54:19.944756 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.944734    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6w2gg"]
Oct 30 21:54:19.947113 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.947090    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:54:19.947308 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.947293    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5khdr\""
Oct 30 21:54:19.947360 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:19.947317    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:54:20.010897 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.010868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-cert\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.010897 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.010901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58xfc\" (UniqueName: \"kubernetes.io/projected/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-kube-api-access-58xfc\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.011083 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.010994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6090-cc3d-4e29-af84-b1de647a037c-config-volume\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.011083 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.011036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.011193 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.011113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795f6090-cc3d-4e29-af84-b1de647a037c-tmp-dir\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.011193 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.011147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph5cp\" (UniqueName: \"kubernetes.io/projected/795f6090-cc3d-4e29-af84-b1de647a037c-kube-api-access-ph5cp\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.046921 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.046889    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:54:20.047070 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.046960    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:54:20.047070 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.047030    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:54:20.047153 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.047107    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-j4tgb\""
Oct 30 21:54:20.111660 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.111583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6090-cc3d-4e29-af84-b1de647a037c-config-volume\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.111660 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.111632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.111856 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.111664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795f6090-cc3d-4e29-af84-b1de647a037c-tmp-dir\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.111856 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.111685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph5cp\" (UniqueName: \"kubernetes.io/projected/795f6090-cc3d-4e29-af84-b1de647a037c-kube-api-access-ph5cp\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.111856 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.111739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-cert\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.111856 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.111764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58xfc\" (UniqueName: \"kubernetes.io/projected/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-kube-api-access-58xfc\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.111856 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.111825    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:54:20.112049 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.111894    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls podName:795f6090-cc3d-4e29-af84-b1de647a037c nodeName:}" failed. No retries permitted until 2026-10-30 21:54:20.611875228 +0000 UTC m=+32.204335392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls") pod "dns-default-vx6lh" (UID: "795f6090-cc3d-4e29-af84-b1de647a037c") : secret "dns-default-metrics-tls" not found
Oct 30 21:54:20.112260 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.112241    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795f6090-cc3d-4e29-af84-b1de647a037c-tmp-dir\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.112517 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.112501    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6090-cc3d-4e29-af84-b1de647a037c-config-volume\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.115865 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.115850    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-cert\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.144671 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.144637    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r28cn"]
Oct 30 21:54:20.166749 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.166719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.182951 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.182921    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph5cp\" (UniqueName: \"kubernetes.io/projected/795f6090-cc3d-4e29-af84-b1de647a037c-kube-api-access-ph5cp\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.184561 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.184536    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58xfc\" (UniqueName: \"kubernetes.io/projected/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-kube-api-access-58xfc\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.211241 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.211217    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r28cn"]
Oct 30 21:54:20.224537 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.224513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:20.294711 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.294685    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:54:20.295160 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.295141    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-plmg6\""
Oct 30 21:54:20.295680 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.295661    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:54:20.295783 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.295699    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:54:20.295783 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.295709    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:54:20.313021 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.312991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ade6beae-e443-46bf-a7fd-3ed608cdd83f-crio-socket\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.313160 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.313044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade6beae-e443-46bf-a7fd-3ed608cdd83f-data-volume\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.313160 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.313126    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.313271 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.313199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qvwd\" (UniqueName: \"kubernetes.io/projected/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-api-access-7qvwd\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.313271 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.313230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.370989 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.370928    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-rk2x9-n9c9p"]
Oct 30 21:54:20.380975 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.380942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.413812 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.413781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade6beae-e443-46bf-a7fd-3ed608cdd83f-data-volume\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.413945 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.413816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.413945 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.413874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qvwd\" (UniqueName: \"kubernetes.io/projected/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-api-access-7qvwd\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.413945 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.413901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.414082 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.413982    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:20.414082 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.413992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ade6beae-e443-46bf-a7fd-3ed608cdd83f-crio-socket\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.414082 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.414054    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:20.914035862 +0000 UTC m=+32.506496031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:20.414226 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.414094    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:20.414226 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.414131    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade6beae-e443-46bf-a7fd-3ed608cdd83f-data-volume\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.414226 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.414151    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:20.914138061 +0000 UTC m=+32.506598233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:20.414226 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.414188    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ade6beae-e443-46bf-a7fd-3ed608cdd83f-crio-socket\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.437564 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.435729    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rk2x9-n9c9p"]
Oct 30 21:54:20.479337 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.479309    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qvwd\" (UniqueName: \"kubernetes.io/projected/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-api-access-7qvwd\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.500643 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.500618    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:54:20.515297 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.515266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.515438 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.515307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.515438 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.515337    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.515548 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.515453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.552473 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.552451    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jwqz7\""
Oct 30 21:54:20.552835 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.552818    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:54:20.616049 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.616164 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.616164 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:20.616164 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.616164 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.616375 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.616246    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:54:20.616375 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.616322    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls podName:795f6090-cc3d-4e29-af84-b1de647a037c nodeName:}" failed. No retries permitted until 2026-10-30 21:54:21.61630365 +0000 UTC m=+33.208763820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls") pod "dns-default-vx6lh" (UID: "795f6090-cc3d-4e29-af84-b1de647a037c") : secret "dns-default-metrics-tls" not found
Oct 30 21:54:20.616565 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616544    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.616630 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.616613    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.617020 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.617002    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.658666 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.658570    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6w2gg"]
Oct 30 21:54:20.661021 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:54:20.660996    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684ddbe7_01fe_42cc_a2c2_ffb9b3cd1e27.slice/crio-acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672 WatchSource:0}: Error finding container acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672: Status 404 returned error can't find the container with id acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672
Oct 30 21:54:20.681806 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.681784    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.693704 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.693679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:20.893228 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.893194    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:20.893391 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.893263    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:20.893391 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.893269    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:20.918350 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.918325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.918680 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.918378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:20.918680 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.918450    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:20.918680 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.918486    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:20.918680 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.918502    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:21.918486858 +0000 UTC m=+33.510947013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:20.918680 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:20.918531    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:21.918519213 +0000 UTC m=+33.510979399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:20.971107 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971084    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:54:20.971107 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971087    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:54:20.971314 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971165    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:54:20.971314 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971193    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:54:20.971314 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971205    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-c7qtj\""
Oct 30 21:54:20.971314 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971221    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-66njg\""
Oct 30 21:54:20.971314 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:20.971208    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-fd64m\""
Oct 30 21:54:21.020927 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.020901    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"]
Oct 30 21:54:21.040102 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.040075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:21.084950 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.084925    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"]
Oct 30 21:54:21.086008 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.085986    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6w2gg" event={"ID":"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27","Type":"ContainerStarted","Data":"acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672"}
Oct 30 21:54:21.088988 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.088960    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rk2x9-n9c9p"]
Oct 30 21:54:21.091033 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:54:21.091008    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b33c42e_93cb_4d53_b533_28c582f874db.slice/crio-b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5 WatchSource:0}: Error finding container b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5: Status 404 returned error can't find the container with id b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5
Oct 30 21:54:21.120474 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.120431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8sdg\" (UniqueName: \"kubernetes.io/projected/d105096e-ef93-47c0-aae7-41e9b2332753-kube-api-access-h8sdg\") pod \"migrator-66df5d496b-cmf99\" (UID: \"d105096e-ef93-47c0-aae7-41e9b2332753\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:21.163137 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.163084    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:54:21.163227 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.163170    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:54:21.191437 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.191387    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hfjcw\""
Oct 30 21:54:21.220758 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.220725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8sdg\" (UniqueName: \"kubernetes.io/projected/d105096e-ef93-47c0-aae7-41e9b2332753-kube-api-access-h8sdg\") pod \"migrator-66df5d496b-cmf99\" (UID: \"d105096e-ef93-47c0-aae7-41e9b2332753\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:21.292920 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.292888    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8sdg\" (UniqueName: \"kubernetes.io/projected/d105096e-ef93-47c0-aae7-41e9b2332753-kube-api-access-h8sdg\") pod \"migrator-66df5d496b-cmf99\" (UID: \"d105096e-ef93-47c0-aae7-41e9b2332753\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:21.364681 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.364653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:21.624196 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.624159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:21.624373 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.624307    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:54:21.624475 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.624379    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls podName:795f6090-cc3d-4e29-af84-b1de647a037c nodeName:}" failed. No retries permitted until 2026-10-30 21:54:23.624358568 +0000 UTC m=+35.216818735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls") pod "dns-default-vx6lh" (UID: "795f6090-cc3d-4e29-af84-b1de647a037c") : secret "dns-default-metrics-tls" not found
Oct 30 21:54:21.758523 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.758492    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"]
Oct 30 21:54:21.761573 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:54:21.761538    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd105096e_ef93_47c0_aae7_41e9b2332753.slice/crio-c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca WatchSource:0}: Error finding container c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca: Status 404 returned error can't find the container with id c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca
Oct 30 21:54:21.772025 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.772005    2576 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rsm8k"]
Oct 30 21:54:21.794137 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.794115    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:21.825448 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.825420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:21.825557 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.825474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:21.825597 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.825578    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:54:21.825652 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.825641    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs podName:b801f58e-2e25-4e89-b099-030d98aa44c9 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:53.825624248 +0000 UTC m=+65.418084417 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs") pod "network-metrics-daemon-k4dcq" (UID: "b801f58e-2e25-4e89-b099-030d98aa44c9") : secret "metrics-daemon-secret" not found
Oct 30 21:54:21.828158 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.828132    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:21.840966 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.840941    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rsm8k"]
Oct 30 21:54:21.849486 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.849462    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 21:54:21.849600 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.849498    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 21:54:21.849600 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.849499    2576 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-sqc6l\""
Oct 30 21:54:21.849600 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.849522    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 21:54:21.849600 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.849524    2576 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 21:54:21.879091 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.879067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.926021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-key\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.926143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcvc\" (UniqueName: \"kubernetes.io/projected/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-kube-api-access-btcvc\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.926232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.926311    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.926319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.926374    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:23.926355701 +0000 UTC m=+35.518815854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.926435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.926470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-cabundle\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.926661    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:21.928148 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:21.926715    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:23.926700976 +0000 UTC m=+35.519161128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:21.929409 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:21.929371    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:22.027525 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.027448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-key\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.027525 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.027501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcvc\" (UniqueName: \"kubernetes.io/projected/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-kube-api-access-btcvc\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.027728 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.027605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-cabundle\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.028346 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.028307    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-cabundle\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.030230 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.030209    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-key\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.089824 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.089781    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99" event={"ID":"d105096e-ef93-47c0-aae7-41e9b2332753","Type":"ContainerStarted","Data":"c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca"}
Oct 30 21:54:22.091596 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.091563    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p" event={"ID":"8b33c42e-93cb-4d53-b533-28c582f874db","Type":"ContainerStarted","Data":"b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5"}
Oct 30 21:54:22.092617 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.092593    2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcvc\" (UniqueName: \"kubernetes.io/projected/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-kube-api-access-btcvc\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.119349 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.119326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:22.156355 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.156326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:22.268044 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.268012    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2xm8j"]
Oct 30 21:54:22.271958 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:54:22.271929    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64e1a7d_50f4_4f15_ab5c_71921ae2af7e.slice/crio-5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e WatchSource:0}: Error finding container 5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e: Status 404 returned error can't find the container with id 5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e
Oct 30 21:54:22.516462 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.516267    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rsm8k"]
Oct 30 21:54:22.519457 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:54:22.519429    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4db1897_c6cc_4c76_810a_e7ed9e00d15e.slice/crio-4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13 WatchSource:0}: Error finding container 4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13: Status 404 returned error can't find the container with id 4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13
Oct 30 21:54:22.566496 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:22.566460    2576 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g96mv"]
Oct 30 21:54:22.570278 ip-10-0-138-166 kubenswrapper[2576]: W1030 21:54:22.570245    2576 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda1d366_0cbe_4eea_a3c3_7c08b2ff04c1.slice/crio-21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c WatchSource:0}: Error finding container 21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c: Status 404 returned error can't find the container with id 21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c
Oct 30 21:54:23.095829 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:23.095759    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2xm8j" event={"ID":"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e","Type":"ContainerStarted","Data":"5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e"}
Oct 30 21:54:23.097088 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:23.097062    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g96mv" event={"ID":"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1","Type":"ContainerStarted","Data":"21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c"}
Oct 30 21:54:23.098224 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:23.098200    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k" event={"ID":"e4db1897-c6cc-4c76-810a-e7ed9e00d15e","Type":"ContainerStarted","Data":"4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13"}
Oct 30 21:54:23.645055 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:23.645014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:23.645227 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:23.645186    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:54:23.645318 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:23.645252    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls podName:795f6090-cc3d-4e29-af84-b1de647a037c nodeName:}" failed. No retries permitted until 2026-10-30 21:54:27.64523358 +0000 UTC m=+39.237693754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls") pod "dns-default-vx6lh" (UID: "795f6090-cc3d-4e29-af84-b1de647a037c") : secret "dns-default-metrics-tls" not found
Oct 30 21:54:23.947389 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:23.947346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:23.947522 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:23.947430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:23.947582 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:23.947551    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:23.947630 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:23.947606    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:27.947589529 +0000 UTC m=+39.540049694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:23.947697 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:23.947664    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:23.947697 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:23.947694    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:27.947684679 +0000 UTC m=+39.540144845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:25.114340 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:25.114250    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99" event={"ID":"d105096e-ef93-47c0-aae7-41e9b2332753","Type":"ContainerStarted","Data":"ae5a2b6d7c20f91f53fd9c20007f42baa78449265abb21c9ee3945283517d638"}
Oct 30 21:54:25.116506 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:25.116473    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6w2gg" event={"ID":"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27","Type":"ContainerStarted","Data":"5e5111b02b2a601baf4a573c642e521219746be87429b4bce52aa3afb3d44872"}
Oct 30 21:54:25.238342 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:25.238285    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99" podStartSLOduration=3.077353058 podStartE2EDuration="5.238270582s" totalImagesPullingTime="2.160917524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:25.237470362 +0000 UTC m=+36.829930536" watchObservedRunningTime="2026-10-30 21:54:25.238270582 +0000 UTC m=+36.830730757"
Oct 30 21:54:25.366134 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:25.366035    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6w2gg" podStartSLOduration=2.203478074 podStartE2EDuration="6.366015679s" totalImagesPullingTime="4.162537605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:25.365053037 +0000 UTC m=+36.957513213" watchObservedRunningTime="2026-10-30 21:54:25.366015679 +0000 UTC m=+36.958475854"
Oct 30 21:54:26.121857 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:26.121819    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p" event={"ID":"8b33c42e-93cb-4d53-b533-28c582f874db","Type":"ContainerStarted","Data":"b99c39a72a008b41b552919d8d4098b625b806be270e249fb2bfaac51def6c8d"}
Oct 30 21:54:27.684068 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:27.683981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:27.684532 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:27.684128    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:54:27.684532 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:27.684195    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls podName:795f6090-cc3d-4e29-af84-b1de647a037c nodeName:}" failed. No retries permitted until 2026-10-30 21:54:35.684177617 +0000 UTC m=+47.276637770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls") pod "dns-default-vx6lh" (UID: "795f6090-cc3d-4e29-af84-b1de647a037c") : secret "dns-default-metrics-tls" not found
Oct 30 21:54:27.986029 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:27.985984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:27.986223 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:27.986098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:27.986223 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:27.986166    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:27.986223 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:27.986193    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:27.986358 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:27.986238    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:35.986219056 +0000 UTC m=+47.578679226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:27.986358 ip-10-0-138-166 kubenswrapper[2576]: E1030 21:54:27.986255    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:35.986248448 +0000 UTC m=+47.578708600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:28.129858 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.129821    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g96mv" event={"ID":"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1","Type":"ContainerStarted","Data":"ca0642a1e25f69692dbcf08b10c36e33ef344823331574bdf31059498c34bbb7"}
Oct 30 21:54:28.130056 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.129969    2576 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:28.131765 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.131739    2576 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k" event={"ID":"e4db1897-c6cc-4c76-810a-e7ed9e00d15e","Type":"ContainerStarted","Data":"8a95f9ad5a9db317f5a64fdd8cedd8bb9567dbe12daa590ec6dc7008c6235481"}
Oct 30 21:54:28.250365 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.250272    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g96mv" podStartSLOduration=34.444595639 podStartE2EDuration="39.250258061s" totalImagesPullingTime="4.805662422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:28.249606212 +0000 UTC m=+39.842066386" watchObservedRunningTime="2026-10-30 21:54:28.250258061 +0000 UTC m=+39.842718236"
Oct 30 21:54:28.250682 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.250661    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p" podStartSLOduration=3.983677458 podStartE2EDuration="8.25065502s" totalImagesPullingTime="4.266977562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:26.247814428 +0000 UTC m=+37.840274600" watchObservedRunningTime="2026-10-30 21:54:28.25065502 +0000 UTC m=+39.843115188"
Oct 30 21:54:28.377183 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.377128    2576 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k" podStartSLOduration=2.764765807 podStartE2EDuration="7.377114479s" totalImagesPullingTime="4.612348672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:28.376762281 +0000 UTC m=+39.969222455" watchObservedRunningTime="2026-10-30 21:54:28.377114479 +0000 UTC m=+39.969574653"
Oct 30 21:54:28.871470 ip-10-0-138-166 kubenswrapper[2576]: I1030 21:54:28.871178    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2r6hv_cebe5fd4-d933-4a1b-a147-e7c013444b0d/dns-node-resolver/0.log"
Oct 30 21:54:28.933518 ip-10-0-138-166 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:54:28.944223 ip-10-0-138-166 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:54:28.944479 ip-10-0-138-166 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:54:28.944539 ip-10-0-138-166 systemd[1]: kubelet.service: Consumed 2.539s CPU time.
Oct 30 21:54:28.954116 ip-10-0-138-166 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:54:29.069101 ip-10-0-138-166 kubenswrapper[4536]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:54:29.069101 ip-10-0-138-166 kubenswrapper[4536]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:54:29.069101 ip-10-0-138-166 kubenswrapper[4536]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:54:29.069101 ip-10-0-138-166 kubenswrapper[4536]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:54:29.071423 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071317    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.071423 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071422    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.071423 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071429    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071432    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071435    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071438    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071441    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071444    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071447    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071449    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071452    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071455    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071457    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071460    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071462    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071466    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071470    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071474    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071485    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071489    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071492    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.071554 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071494    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071497    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071499    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071502    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071505    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071507    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071510    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071513    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071516    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071518    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071521    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071523    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071526    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071528    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071531    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071533    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071536    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071538    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071540    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071543    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.072058 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071546    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071548    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071551    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071554    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071556    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071559    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071562    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071564    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071567    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071571    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071573    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071576    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071580    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071583    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071585    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071588    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071590    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071594    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071596    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.072592 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071599    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071602    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071605    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071607    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071610    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071612    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071615    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071618    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071621    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071623    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071625    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071628    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071631    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071634    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071636    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071639    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071641    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071643    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071646    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071648    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.073071 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071650    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071653    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071656    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071659    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071661    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071663    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071666    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071669    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071671    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071676    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071680    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071684    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071687    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071689    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071692    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071694    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071697    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071699    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071702    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.073597 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071704    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071707    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071710    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071713    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071715    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071718    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.071721    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072509    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072517    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072520    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072523    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072525    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072528    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072531    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072535    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072538    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072540    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072543    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072545    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.074089 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072548    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072552    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072556    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072559    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072562    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072565    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072567    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072570    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072573    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072575    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072578    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072580    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072583    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072586    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072588    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072591    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072593    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072596    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072599    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.074585 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072601    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072604    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072607    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072610    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072613    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072616    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072618    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072621    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072623    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072626    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072628    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072631    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072633    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072636    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072638    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072641    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072643    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072646    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072648    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072651    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.075074 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072654    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072656    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072659    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072662    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072664    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072667    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072669    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072672    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072674    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072677    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072679    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072682    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072684    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072687    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072691    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072694    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072698    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072700    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072702    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072705    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.075602 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072708    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072711    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072713    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072716    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072718    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072721    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072725    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072728    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072731    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072733    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072736    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072738    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072741    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072743    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072746    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072748    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072751    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072753    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072756    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.076098 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072758    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072761    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072764    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072766    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072769    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072771    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072774    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072776    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072779    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072782    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072785    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072787    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072790    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072792    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072795    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.072798    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073269    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073274    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073277    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.076599 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073280    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073282    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073285    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073287    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073290    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073293    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073296    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073298    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073301    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073303    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073305    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073308    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073311    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073313    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073316    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073318    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073321    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073324    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073326    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073329    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.077068 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073332    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073337    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073340    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073343    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073345    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073348    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073350    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073355    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073359    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073361    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073364    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073366    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073369    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073371    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073374    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073376    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073379    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073381    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073384    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.077662 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073386    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073389    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073408    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073412    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073415    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073417    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073420    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073423    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073425    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073428    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073430    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073432    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073435    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073437    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073440    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073443    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073445    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073449    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073451    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073454    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.078135 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073456    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073459    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073462    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073464    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073467    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073469    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073472    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073474    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073477    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073479    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073482    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073484    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073487    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073489    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073491    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073494    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073497    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073499    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073502    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.078634 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073504    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073506    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073509    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073511    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073514    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073516    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073519    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073522    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073524    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073527    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073530    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073532    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073534    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073537    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073543    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073545    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073547    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073550    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073553    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.079100 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073555    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073558    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073560    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073563    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073566    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.073569    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073665    4536 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073672    4536 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073678    4536 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073683    4536 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073687    4536 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073691    4536 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073695    4536 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073699    4536 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073702    4536 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073705    4536 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073709    4536 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073712    4536 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073715    4536 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073718    4536 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073720    4536 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073724    4536 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073726    4536 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:54:29.079754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073730    4536 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073734    4536 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073737    4536 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073740    4536 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073745    4536 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073748    4536 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073753    4536 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073755    4536 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073758    4536 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073762    4536 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073765    4536 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073768    4536 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073771    4536 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073774    4536 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073777    4536 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073782    4536 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073785    4536 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073788    4536 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073790    4536 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073794    4536 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073797    4536 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073801    4536 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073804    4536 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073806    4536 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:54:29.080740 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073809    4536 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073812    4536 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073817    4536 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073820    4536 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073823    4536 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073825    4536 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073828    4536 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073831    4536 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073834    4536 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073837    4536 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073840    4536 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073843    4536 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073847    4536 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073852    4536 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073855    4536 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073858    4536 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073861    4536 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073864    4536 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073867    4536 flags.go:64] FLAG: --help="false"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073870    4536 flags.go:64] FLAG: --hostname-override="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073874    4536 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073877    4536 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073880    4536 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:54:29.082070 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073883    4536 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073886    4536 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073889    4536 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073892    4536 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073895    4536 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073898    4536 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073900    4536 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073903    4536 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073906    4536 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073909    4536 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073912    4536 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073915    4536 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073917    4536 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073920    4536 flags.go:64] FLAG: --lock-file=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073923    4536 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073925    4536 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073928    4536 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073934    4536 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073936    4536 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073940    4536 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073943    4536 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073948    4536 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073951    4536 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073954    4536 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073957    4536 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:54:29.083175 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073961    4536 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073964    4536 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073969    4536 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073972    4536 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073975    4536 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073978    4536 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073981    4536 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073984    4536 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073986    4536 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073990    4536 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.073997    4536 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074000    4536 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074003    4536 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074006    4536 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074009    4536 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074012    4536 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074015    4536 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074018    4536 flags.go:64] FLAG: --port="10250"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074021    4536 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074024    4536 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-06b25911203d41eed"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074027    4536 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074030    4536 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074033    4536 flags.go:64] FLAG: --register-node="true"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074036    4536 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074039    4536 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074042    4536 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:54:29.084229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074045    4536 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074048    4536 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074054    4536 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074057    4536 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074060    4536 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074063    4536 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074066    4536 flags.go:64] FLAG: --runonce="false"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074069    4536 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074072    4536 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074075    4536 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074078    4536 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074081    4536 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074084    4536 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074087    4536 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074091    4536 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074094    4536 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074097    4536 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074100    4536 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074103    4536 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074106    4536 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074109    4536 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074112    4536 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074117    4536 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074120    4536 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074123    4536 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:54:29.085453 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074129    4536 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074132    4536 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074135    4536 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074138    4536 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074141    4536 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074144    4536 flags.go:64] FLAG: --v="2"
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074149    4536 flags.go:64] FLAG: --version="false"
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074153    4536 flags.go:64] FLAG: --vmodule=""
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074157    4536 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074162    4536 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074292    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074296    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074299    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074301    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074304    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074306    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074309    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074312    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074314    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074317    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074319    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.086219 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074322    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074325    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074327    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074330    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074332    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074335    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074337    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074340    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074342    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074345    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074347    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074352    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074355    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074357    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074360    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074362    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074365    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074368    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074370    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074373    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.086981 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074376    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074380    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074382    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074385    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074387    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074389    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074409    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074412    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074415    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074417    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074420    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074423    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074425    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074428    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074430    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074433    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074435    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074438    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074440    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074443    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.087665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074445    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074448    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074450    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074455    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074458    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074460    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074463    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074467    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074471    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074474    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074477    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074479    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074483    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074486    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074489    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074491    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074494    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074496    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074499    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.088145 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074502    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074504    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074507    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074509    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074512    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074514    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074517    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074519    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074522    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074525    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074528    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074530    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074533    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074535    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074538    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074540    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074544    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074546    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074548    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.088698 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074551    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074553    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074557    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074561    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074564    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074567    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074572    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074574    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074577    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074581    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074583    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074586    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074589    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074591    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074594    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074596    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.089278 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074599    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.074626    4536 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074659    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074664    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074667    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074670    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074673    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074675    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074679    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074681    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074684    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074687    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074690    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074694    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074696    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074698    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.089768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074701    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074704    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074706    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074708    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074711    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074713    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074716    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074720    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074722    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074725    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074728    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074731    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074733    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074736    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074739    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074741    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074744    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074746    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074749    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074751    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.090160 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074754    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074756    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074758    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074761    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074764    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074766    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074769    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074771    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074774    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074777    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074780    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074782    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074785    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074787    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074790    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074792    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074795    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074798    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074800    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074804    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.090709 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074806    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074809    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074811    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074814    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074817    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074821    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074824    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074827    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074830    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074832    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074835    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074837    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074840    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074842    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074845    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074847    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074850    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074852    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074855    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.091196 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074857    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074860    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074863    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074866    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074868    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074871    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074873    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074876    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074878    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074881    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074883    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074886    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074890    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074892    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074895    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074897    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074900    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074903    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074905    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.091710 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074910    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074914    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074917    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074920    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074922    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074925    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074927    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074930    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074932    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074935    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074937    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074940    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074942    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.074945    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.078743    4536 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.078756    4536 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.078770    4536 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.078775    4536 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078832    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.092162 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078837    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078840    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078843    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078846    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078849    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078852    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078855    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078858    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078860    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078863    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078865    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078869    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078871    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078874    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078877    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078880    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078882    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078885    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078887    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.092667 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078890    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078893    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078896    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078898    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078901    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078903    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078907    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078909    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078912    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078915    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078917    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078920    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078922    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078925    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078927    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078929    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078932    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078935    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078937    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078941    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.093147 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078945    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078948    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078950    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078953    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078955    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078958    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078962    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078964    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078967    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078969    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078972    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078975    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078977    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078980    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078983    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078985    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078988    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078990    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078993    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.093664 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078996    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.078998    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079001    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079004    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079006    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079009    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079012    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079014    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079017    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079020    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079022    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079024    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079027    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079029    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079032    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079034    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079036    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079039    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079041    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079044    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.094123 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079047    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079049    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079052    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079055    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079058    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079060    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079063    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079065    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079068    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079071    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079073    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079075    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079078    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079080    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079083    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079085    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079088    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079090    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079093    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079095    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.094660 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079098    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079100    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079103    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079105    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079108    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079111    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079115    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.079142    4536 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079467    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079481    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079487    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079491    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079502    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079507    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079512    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.095187 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079516    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079521    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079525    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079530    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079534    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079539    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079543    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079547    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079551    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079566    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079573    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079577    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079581    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079585    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079591    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079596    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079600    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079604    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079609    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079613    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.095768 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079618    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079622    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079631    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079635    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079639    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079643    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079648    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079651    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079655    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079660    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079665    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079670    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079674    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079678    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079683    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079691    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079696    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079700    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079704    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.096455 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079708    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079712    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079716    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079720    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079724    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079728    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079732    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079736    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079745    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079749    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079753    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079757    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079761    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079766    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079770    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079775    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079780    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079785    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079789    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.097159 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079793    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079798    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079807    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079811    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079816    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079821    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079825    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079829    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079833    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079837    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079842    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079846    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079850    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079854    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079858    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079866    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079871    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079875    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079879    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079883    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.097653 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079887    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079891    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079895    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079900    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079904    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079908    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079912    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079921    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079925    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079929    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079933    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079940    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079946    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079951    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079957    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079963    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079968    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079973    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079978    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079988    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.098170 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.079993    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.080050    4536 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080166    4536 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080173    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080177    4536 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080181    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080186    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080190    4536 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080194    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080198    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080202    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080206    4536 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080211    4536 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080215    4536 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080219    4536 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080228    4536 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:29.098956 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080232    4536 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080237    4536 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080241    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080245    4536 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080249    4536 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080253    4536 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080258    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080261    4536 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080265    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080270    4536 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080273    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080338    4536 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080365    4536 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080372    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080415    4536 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080568    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080578    4536 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080585    4536 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080590    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:29.099611 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080594    4536 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080598    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080602    4536 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080607    4536 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080611    4536 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080615    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080620    4536 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080624    4536 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080628    4536 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080632    4536 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080636    4536 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080640    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080644    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080647    4536 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080651    4536 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080656    4536 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080660    4536 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080664    4536 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080669    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080677    4536 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:29.100193 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080682    4536 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080686    4536 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080691    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080697    4536 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080702    4536 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080707    4536 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080712    4536 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080717    4536 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080722    4536 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080727    4536 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080734    4536 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080742    4536 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080747    4536 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080753    4536 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080761    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080767    4536 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080772    4536 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080776    4536 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080780    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:29.100976 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080785    4536 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080789    4536 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080793    4536 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080797    4536 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080802    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080806    4536 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080811    4536 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080816    4536 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080820    4536 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080825    4536 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080829    4536 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080833    4536 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080837    4536 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080841    4536 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080846    4536 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080850    4536 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080855    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080859    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080863    4536 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:29.101804 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080868    4536 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080873    4536 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080878    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080884    4536 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080889    4536 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080894    4536 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080898    4536 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080903    4536 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080908    4536 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080912    4536 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080916    4536 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080923    4536 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080927    4536 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080931    4536 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:29.080935    4536 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.081027    4536 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.082580    4536 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.082657    4536 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.082936    4536 server.go:1014] "Starting client certificate rotation"
Oct 30 21:54:29.102665 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.083061    4536 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.083102    4536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:48:49 +0000 UTC" deadline="2028-04-21 14:02:37.217682666 +0000 UTC"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.083137    4536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12928h8m8.13454744s"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.083690    4536 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.085426    4536 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.086941    4536 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.088455    4536 log.go:25] "Validated CRI v1 image API"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.089160    4536 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:54:29.103579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.090700    4536 fs.go:135] Filesystem UUIDs: map[703f5809-d18f-477d-8c75-6df7d9e4cc21:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f1836b2b-b7b1-4d97-91b8-9b6e595017d4:/dev/nvme0n1p3]
Oct 30 21:54:29.103976 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.090719    4536 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/12c3c632cb2610779d1d7078a0da4366d394e1a9df4d3d0624c84f3d2daddd72/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/12c3c632cb2610779d1d7078a0da4366d394e1a9df4d3d0624c84f3d2daddd72/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c/userdata/shm major:0 minor:225 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13/userdata/shm major:0 minor:223 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232/userdata/shm major:0 minor:71 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06a83215-f446-480d-9036-69d4ac88e172/volumes/kubernetes.io~projected/kube-api-access-8sdz9:{mountpoint:/var/lib/kubelet/pods/06a83215-f446-480d-9036-69d4ac88e172/volumes/kubernetes.io~projected/kube-api-access-8sdz9 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/29c88d96-2e8f-4431-9975-f4e8ecba95ba/volumes/kubernetes.io~projected/kube-api-access-j2rm6:{mountpoint:/var/lib/kubelet/pods/29c88d96-2e8f-4431-9975-f4e8ecba95ba/volumes/kubernetes.io~projected/kube-api-access-j2rm6 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volumes/kubernetes.io~projected/kube-api-access-89lw7:{mountpoint:/var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volumes/kubernetes.io~projected/kube-api-access-89lw7 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46124db8-6b86-4a74-8ef7-80a2a04a8463/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/46124db8-6b86-4a74-8ef7-80a2a04a8463/volumes/kubernetes.io~secret/agent-certs major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/volumes/kubernetes.io~projected/kube-api-access-58xfc:{mountpoint:/var/lib/kubelet/pods/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/volumes/kubernetes.io~projected/kube-api-access-58xfc major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/795f6090-cc3d-4e29-af84-b1de647a037c/volumes/kubernetes.io~projected/kube-api-access-ph5cp:{mountpoint:/var/lib/kubelet/pods/795f6090-cc3d-4e29-af84-b1de647a037c/volumes/kubernetes.io~projected/kube-api-access-ph5cp major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80c062cd-a506-47bf-ad60-4bcfe3c5a19e/volumes/kubernetes.io~projected/kube-api-access-tljcv:{mountpoint:/var/lib/kubelet/pods/80c062cd-a506-47bf-ad60-4bcfe3c5a19e/volumes/kubernetes.io~projected/kube-api-access-tljcv major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/834c4daa-eb50-46b5-919f-c201d9458e11/volumes/kubernetes.io~projected/kube-api-access-cjd6n:{mountpoint:/var/lib/kubelet/pods/834c4daa-eb50-46b5-919f-c201d9458e11/volumes/kubernetes.io~projected/kube-api-access-cjd6n major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b33c42e-93cb-4d53-b533-28c582f874db/volumes/kubernetes.io~projected/kube-api-access-qrkcm:{mountpoint:/var/lib/kubelet/pods/8b33c42e-93cb-4d53-b533-28c582f874db/volumes/kubernetes.io~projected/kube-api-access-qrkcm major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~projected/kube-api-access-fbn98:{mountpoint:/var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~projected/kube-api-access-fbn98 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ade6beae-e443-46bf-a7fd-3ed608cdd83f/volumes/kubernetes.io~projected/kube-api-access-7qvwd:{mountpoint:/var/lib/kubelet/pods/ade6beae-e443-46bf-a7fd-3ed608cdd83f/volumes/kubernetes.io~projected/kube-api-access-7qvwd major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b801f58e-2e25-4e89-b099-030d98aa44c9/volumes/kubernetes.io~projected/kube-api-access-76xfx:{mountpoint:/var/lib/kubelet/pods/b801f58e-2e25-4e89-b099-030d98aa44c9/volumes/kubernetes.io~projected/kube-api-access-76xfx major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cebe5fd4-d933-4a1b-a147-e7c013444b0d/volumes/kubernetes.io~projected/kube-api-access-cmk4b:{mountpoint:/var/lib/kubelet/pods/cebe5fd4-d933-4a1b-a147-e7c013444b0d/volumes/kubernetes.io~projected/kube-api-access-cmk4b major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d105096e-ef93-47c0-aae7-41e9b2332753/volumes/kubernetes.io~projected/kube-api-access-h8sdg:{mountpoint:/var/lib/kubelet/pods/d105096e-ef93-47c0-aae7-41e9b2332753/volumes/kubernetes.io~projected/kube-api-access-h8sdg major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4db1897-c6cc-4c76-810a-e7ed9e00d15e/volumes/kubernetes.io~projected/kube-api-access-btcvc:{mountpoint:/var/lib/kubelet/pods/e4db1897-c6cc-4c76-810a-e7ed9e00d15e/volumes/kubernetes.io~projected/kube-api-access-btcvc major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e4db1897-c6cc-4c76-810a-e7ed9e00d15e/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/e4db1897-c6cc-4c76-810a-e7ed9e00d15e/volumes/kubernetes.io~secret/signing-key major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95/volumes/kubernetes.io~projected/kube-api-access-zwkrc:{mountpoint:/var/lib/kubelet/pods/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95/volumes/kubernetes.io~projected/kube-api-access-zwkrc major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1/volumes/kubernetes.io~projected/kube-api-access-95k2j:{mountpoint:/var/lib/kubelet/pods/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1/volumes/kubernetes.io~projected/kube-api-access-95k2j major:0 minor:220 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/b086a7798c25bda7b857e4e3300c5ce3543ae5bf58a2fd6208b07f6cc71e906e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/8a94b85d70ec1c96c0da53f532f58f4ce78cb79a1b83acc4eed44ecb39ac9e57/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/9867936d75625aacf6e9cb0b44f1ed872a00569fd2fac0af161d1be2f2f8c21c/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-110:{mountpoint:/var/lib/containers/storage/overlay/eb856077a3ade5748c457339085360b81c97c3de3c994da59ada898b750d02bf/merged major:0 minor:110 fsType:overlay blockSize:0} overlay_0-115:{mountpoint:/var/lib/containers/storage/overlay/93f1dec6ffa2e07de2ff24afe1055fa71b5c25c01f8d20115db5f935f9b8af96/merged major:0 minor:115 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/a7c1f810bd063ab6fd7db5bc96d2fcd47636fdbc0c1bf42b8756784916071d64/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/06c3ea1d15a65b98b9d4da4f26b1e6c1ef02520dc46e490aa34872a350fcc09f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/2edd1c15fcb4c3c3bbd6cc002bfa456b6e46dd449acd02c7855fca57549f0c0a/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/48fbf6e0ef6367560d88f433aa1c16355613b4fd321f5639ba9ba1fc02f50e2c/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/5e8520e7c8c7b2680d3bcc52bcb032c0b6c859a7303cc68a47ce9b2c01950b96/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/1022325485d87a40d1677c4ffb573a1c57fbb4fe5e9e7288932dbceeef9bc166/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/b6a2a2f2813b5f9133a2d3c4ba406f8c4be79b74dbc37cf24e7ffddb8eab1517/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-153:{mountpoint:/var/lib/containers/storage/overlay/99bbb14b1226456c266e8ca49b7b2f30a1bd6fd5213ea4fd218a5ccfe739ff59/merged major:0 minor:153 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/e8fc8dd479f10d65cfcf71173791d425d7d0e549145260757c90031746ec5496/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/3d5dba33424964d9f62610fb5f284b9772d12d5237d228a94ad680ce7509855d/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/7c904cdd0419c0acb06f070702192fff266acc9caff1e7a16fb9d9ca2b705d6b/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/595831e8e1eaa14b19be78483e1d0bdea74d0b6285c0386b11390e4335e7a140/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/23060cc09f43642a2b10b4df716c98175ecf89b5b008ac8955a4f009544553d8/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/44f9b8d08a7cffbe717cd80dc466d7db7a25fe6601dec8147e03d372682e8214/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/0371d53f42fd9670907ef0aa17ea20383637c1f84a5acbc961f06e35519ed1cb/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/8629c3c754afcb74a48f328172fa703021833aa2c50daa74a787c85dd0cbe6e7/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-205:{mountpoint:/var/lib/containers/storage/overlay/221c662d511a6fbea1bbbf6261f01de65a209b8584d562130e754f739b5dfdf4/merged major:0 minor:205 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/83d553673a8737a3621848c683345436d11c660c5d5421aad81f707d7333d8a0/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/c1e8859ae61ad1d69809a8bb7064574d7dc814ea85e7e1b326b184b2fef58cfe/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/bd986fe07187ee260c3d68951d424aa646d85af11dda78af3f02b64a80f777b4/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-229:{mountpoint:/var/lib/containers/storage/overlay/abc176993c79599c254d377af3e7f618c24386de9caa8697ae8aa3a3f547fc9c/merged major:0 minor:229 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/f24f9ff7630d4e4d41743bb4a1202f63c084c92b02c7050b8e28df3a0c43bd14/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/6fb99644b3999556688fd82d7ece8c7b6d0e3d0745496d1f1d774054b13598fb/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/69bcfa52da8d40bb835b6c0a239e5fc2003e1cf9752f8901b93f85979297e108/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/b8c5ce045c8ff60bd8401beaf0e6f3ae1ae8475c04630f36114e341df55bc886/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/65a3d4105bc523beaaf075d4412996ac3659e5103554e44de287c4155f870d1b/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/a69b63dfe2b322713edc5303e937737594fde5f894f839de974a2f99f23943db/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/f9eb3227eb5101deb91f54ebe2e37561f49570e661d42bea7ba6e2f26f305907/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/cf5deb2266ea8bda05b53f904fdb63c32744c58d5d9e03911bed0d117eb12bc3/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/706d8eefc9614bac0a4e4aa6a042018a0ad2f733a2e6bc55bf92a38394a2c404/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/012f97f28fd3cd41442a7ad77e8d870d6ed2ecf81abb5b85b9716a359c58a447/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/46c88dd405a4d170259c98281b38a1a60b49fbf284a1417680bc54069558f2c6/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/de5e8f6bb0ea7ea1e76f5954325176627bfb5e1681762cc3e356e7cce1852d01/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/070c27114a9a521a13491df603c5afcb3a886d108e1b74ff86f2cb50040143df/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/a1f2129fb89dc708d1a6f1347e752c869a38b325d35b98bb8bd2e57c6dbcd437/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/0956ca2c98aca86f9ef13082f03bbc8ff0097102ecf27c4a535fe50ebfe4fbef/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/f9a1530bd3041265ecdeecbaa3e441aa355592976780d5c47554fba711f04f84/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/bf8cf1c37b00ce16c926237e98f1feba855beb6958d3c5070c11eac344241441/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.097542    4536 manager.go:217] Machine: {Timestamp:2026-10-30 21:54:29.097067726 +0000 UTC m=+0.084875979 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097281 MemoryCapacity:32812007424 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f7568829fbf8dde6767b4a670268b SystemUUID:ec2f7568-829f-bf8d-de67-67b4a670268b BootID:1462693b-d101-453c-a329-02046e5e5229 Filesystems:[{Device:/var/lib/kubelet/pods/cebe5fd4-d933-4a1b-a147-e7c013444b0d/volumes/kubernetes.io~projected/kube-api-access-cmk4b DeviceMajor:0 DeviceMinor:58 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/volumes/kubernetes.io~projected/kube-api-access-58xfc DeviceMajor:0 DeviceMinor:201 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c841945b9bcf7c9e04d6e9ffba71ca44dfd71c37fa3857a40d4b7fcfe55631ca/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/21de329564263e4f242a70b88c4cf60f2264abb5a24298b8f44ca9f4e46ff23c/userdata/shm DeviceMajor:0 DeviceMinor:225 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~projected/kube-api-access-fbn98 DeviceMajor:0 DeviceMinor:51 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/77a7a9d1dc671c06d741cf7c4f97f98051fcd2d9949510beafaf61c16f34070b/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232/userdata/shm DeviceMajor:0 DeviceMinor:71 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/177fcf1f4a82c3063f9742ed1d7b21c9394c1a308f3efc40e67b84bbaf88fd41/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/12c3c632cb2610779d1d7078a0da4366d394e1a9df4d3d0624c84f3d2daddd72/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/834c4daa-eb50-46b5-919f-c201d9458e11/volumes/kubernetes.io~projected/kube-api-access-cjd6n DeviceMajor:0 DeviceMinor:56 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-110 DeviceMajor:0 DeviceMinor:110 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/795f6090-cc3d-4e29-af84-b1de647a037c/volumes/kubernetes.io~projected/kube-api-access-ph5cp DeviceMajor:0 DeviceMinor:197 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ade6beae-e443-46bf-a7fd-3ed608cdd83f/volumes/kubernetes.io~projected/kube-api-access-7qvwd DeviceMajor:0 DeviceMinor:204 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/29c88d96-2e8f-4431-9975-f4e8ecba95ba/volumes/kubernetes.io~projected/kube-api-access-j2rm6 DeviceMajor:0 DeviceMinor:59 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/01bf288dd413f1571326339b8c887201184918d1bafbc04d7894659febd26442/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-205 DeviceMajor:0 DeviceMinor:205 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b33c42e-93cb-4d53-b533-28c582f874db/volumes/kubernetes.io~projected/kube-api-access-qrkcm DeviceMajor:0 DeviceMinor:207 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4857bb6014c82e7e9f877be0d2c7a95464e21bcb3c49b7dbde3aaf9a1f799c13/userdata/shm DeviceMajor:0 DeviceMinor:223 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95/volumes/kubernetes.io~projected/kube-api-access-zwkrc DeviceMajor:0 DeviceMinor:55 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/33857ca2a90533883e5e766331b56a06f937778452004c84393c754cdfca1927/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d105096e-ef93-47c0-aae7-41e9b2332753/volumes/kubernetes.io~projected/kube-api-access-h8sdg DeviceMajor:0 DeviceMinor:212 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e4db1897-c6cc-4c76-810a-e7ed9e00d15e/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:221 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0b072a91761a2d6a64bc1d528c5ab1e9627d32adf8c261127945e4db3c5d95a6/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e4db1897-c6cc-4c76-810a-e7ed9e00d15e/volumes/kubernetes.io~projected/kube-api-access-btcvc DeviceMajor:0 DeviceMinor:222 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volumes/kubernetes.io~projected/kube-api-access-89lw7 DeviceMajor:0 DeviceMinor:57 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/acf9e0bd60772d8d282833a8bc7eeb0a93a426e703398bb5183ba56a3dc4e672/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/94ac4f3c-9331-4223-bad5-dd4299321eb6/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:50 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b732b59f76440050b51c1a7e6054a8eac71165ac217877cd888cd5f8ab861125/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-115 DeviceMajor:0 DeviceMinor:115 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-229 DeviceMajor:0 DeviceMinor:229 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/044e07f07d8e865225a7d8f4cce5dc884c335a6f461f003c2b478dc29058d79e/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2be95b15-43a7-4f7e-85c7-14a0adccb875/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406003712 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/b801f58e-2e25-4e89-b099-030d98aa44c9/volumes/kubernetes.io~projected/kube-api-access-76xfx DeviceMajor:0 DeviceMinor:54 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:217 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5849cc3ddc2f5e1426b2ad00525db481fa15bd2594fd824f34a8b2bd2776224e/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1/volumes/kubernetes.io~projected/kube-api-access-95k2j DeviceMajor:0 DeviceMinor:220 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/06a83215-f446-480d-9036-69d4ac88e172/volumes/kubernetes.io~projected/kube-api-access-8sdz9 DeviceMajor:0 DeviceMinor:52 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-153 DeviceMajor:0 DeviceMinor:153 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46124db8-6b86-4a74-8ef7-80a2a04a8463/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:48 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/80c062cd-a506-47bf-ad60-4bcfe3c5a19e/volumes/kubernetes.io~projected/kube-api-access-tljcv DeviceMajor:0 DeviceMinor:53 Capacity:31633408000 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:21de329564263e4 MacAddress:22:d9:34:47:2e:cf Speed:10000 Mtu:8901} {Name:4857bb6014c82e7 MacAddress:3e:2d:10:19:ae:a6 Speed:10000 Mtu:8901} {Name:5849cc3ddc2f5e1 MacAddress:56:e4:b1:0f:98:c6 Speed:10000 Mtu:8901} {Name:acf9e0bd60772d8 MacAddress:2e:80:a8:23:65:94 Speed:10000 Mtu:8901} {Name:b679ec01587cf78 MacAddress:1e:e8:f2:28:a4:21 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:75:64:c2:22:c1 Speed:0 Mtu:9001} {Name:br-int MacAddress:2a:09:2f:34:6e:76 Speed:0 Mtu:8901} {Name:c841945b9bcf7c9 MacAddress:ce:0f:c7:7b:43:2f Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:75:64:c2:22:c1 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:a6:b6:10:f5:9d:f6 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ae:d1:ea:c3:c1:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812007424 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.097787    4536 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.097828    4536 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.098110    4536 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.098132    4536 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-166.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.098947    4536 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.098962    4536 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.098978    4536 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.099004    4536 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.099125    4536 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.099694    4536 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.099914    4536 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100128    4536 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100145    4536 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100156    4536 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100165    4536 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100174    4536 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100855    4536 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.100867    4536 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:54:29.104222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.101443    4536 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104348    4536 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104362    4536 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104502    4536 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104760    4536 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104806    4536 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104960    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104977    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104987    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.104997    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105007    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105016    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105024    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105033    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105058    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105069    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105092    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105107    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105177    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:54:29.105421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.105189    4536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:54:29.108824 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.108807    4536 server.go:1263] "Started kubelet"
Oct 30 21:54:29.108940 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.108896    4536 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:54:29.108993 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.108964    4536 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:54:29.109276 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.109133    4536 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:54:29.109466 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.109451    4536 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:54:29.109662 ip-10-0-138-166 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:54:29.110643 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.110629    4536 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:54:29.111945 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.111926    4536 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:54:29.111945 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.111935    4536 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.112308    4536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:48:50 +0000 UTC" deadline="2028-05-24 23:33:15.923593548 +0000 UTC"
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.112352    4536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13729h38m46.81124472s"
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.112496    4536 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-166.us-west-1.compute.internal\" not found"
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.112140    4536 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.112610    4536 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.115514    4536 factory.go:55] Registering systemd factory
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.115533    4536 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.116607    4536 factory.go:153] Registering CRI-O factory
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.116623    4536 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.116675    4536 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.116697    4536 factory.go:103] Registering Raw factory
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.116715    4536 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:54:29.118239 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.117031    4536 manager.go:319] Starting recovery of all containers
Oct 30 21:54:29.118916 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.118870    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="795f6090-cc3d-4e29-af84-b1de647a037c" volumeName="kubernetes.io/empty-dir/795f6090-cc3d-4e29-af84-b1de647a037c-tmp-dir" seLinuxMountContext=""
Oct 30 21:54:29.119006 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.118920    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="834c4daa-eb50-46b5-919f-c201d9458e11" volumeName="kubernetes.io/projected/834c4daa-eb50-46b5-919f-c201d9458e11-kube-api-access-cjd6n" seLinuxMountContext=""
Oct 30 21:54:29.119006 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.118945    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ade6beae-e443-46bf-a7fd-3ed608cdd83f" volumeName="kubernetes.io/projected/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-api-access-7qvwd" seLinuxMountContext=""
Oct 30 21:54:29.119006 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.118958    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cebe5fd4-d933-4a1b-a147-e7c013444b0d" volumeName="kubernetes.io/projected/cebe5fd4-d933-4a1b-a147-e7c013444b0d-kube-api-access-cmk4b" seLinuxMountContext=""
Oct 30 21:54:29.119006 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.118999    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d105096e-ef93-47c0-aae7-41e9b2332753" volumeName="kubernetes.io/projected/d105096e-ef93-47c0-aae7-41e9b2332753-kube-api-access-h8sdg" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119012    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06a83215-f446-480d-9036-69d4ac88e172" volumeName="kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119036    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29c88d96-2e8f-4431-9975-f4e8ecba95ba" volumeName="kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119053    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29c88d96-2e8f-4431-9975-f4e8ecba95ba" volumeName="kubernetes.io/projected/29c88d96-2e8f-4431-9975-f4e8ecba95ba-kube-api-access-j2rm6" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119070    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b33c42e-93cb-4d53-b533-28c582f874db" volumeName="kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119087    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4db1897-c6cc-4c76-810a-e7ed9e00d15e" volumeName="kubernetes.io/secret/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-key" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119100    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46124db8-6b86-4a74-8ef7-80a2a04a8463" volumeName="kubernetes.io/secret/46124db8-6b86-4a74-8ef7-80a2a04a8463-agent-certs" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119117    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80c062cd-a506-47bf-ad60-4bcfe3c5a19e" volumeName="kubernetes.io/projected/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kube-api-access-tljcv" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119135    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b33c42e-93cb-4d53-b533-28c582f874db" volumeName="kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119150    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="834c4daa-eb50-46b5-919f-c201d9458e11" volumeName="kubernetes.io/configmap/834c4daa-eb50-46b5-919f-c201d9458e11-serviceca" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119173    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46124db8-6b86-4a74-8ef7-80a2a04a8463" volumeName="kubernetes.io/configmap/46124db8-6b86-4a74-8ef7-80a2a04a8463-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:54:29.119192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119184    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06a83215-f446-480d-9036-69d4ac88e172" volumeName="kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119253    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be95b15-43a7-4f7e-85c7-14a0adccb875" volumeName="kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-env-overrides" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119269    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be95b15-43a7-4f7e-85c7-14a0adccb875" volumeName="kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119287    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27" volumeName="kubernetes.io/projected/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-kube-api-access-58xfc" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119300    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="795f6090-cc3d-4e29-af84-b1de647a037c" volumeName="kubernetes.io/configmap/795f6090-cc3d-4e29-af84-b1de647a037c-config-volume" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119321    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d64e1a7d-50f4-4f15-ab5c-71921ae2af7e" volumeName="kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119334    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b33c42e-93cb-4d53-b533-28c582f874db" volumeName="kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119354    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ac4f3c-9331-4223-bad5-dd4299321eb6" volumeName="kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-tuned" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119373    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ac4f3c-9331-4223-bad5-dd4299321eb6" volumeName="kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-tmp" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119384    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cebe5fd4-d933-4a1b-a147-e7c013444b0d" volumeName="kubernetes.io/empty-dir/cebe5fd4-d933-4a1b-a147-e7c013444b0d-tmp-dir" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119437    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4db1897-c6cc-4c76-810a-e7ed9e00d15e" volumeName="kubernetes.io/projected/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-kube-api-access-btcvc" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119453    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95" volumeName="kubernetes.io/projected/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-kube-api-access-zwkrc" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119464    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1" volumeName="kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119503    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be95b15-43a7-4f7e-85c7-14a0adccb875" volumeName="kubernetes.io/projected/2be95b15-43a7-4f7e-85c7-14a0adccb875-kube-api-access-89lw7" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119521    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29c88d96-2e8f-4431-9975-f4e8ecba95ba" volumeName="kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119534    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27" volumeName="kubernetes.io/secret/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-cert" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119550    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="795f6090-cc3d-4e29-af84-b1de647a037c" volumeName="kubernetes.io/projected/795f6090-cc3d-4e29-af84-b1de647a037c-kube-api-access-ph5cp" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119564    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b33c42e-93cb-4d53-b533-28c582f874db" volumeName="kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119579    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ade6beae-e443-46bf-a7fd-3ed608cdd83f" volumeName="kubernetes.io/empty-dir/ade6beae-e443-46bf-a7fd-3ed608cdd83f-data-volume" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119597    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95" volumeName="kubernetes.io/configmap/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119611    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="29c88d96-2e8f-4431-9975-f4e8ecba95ba" volumeName="kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119629    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e4db1897-c6cc-4c76-810a-e7ed9e00d15e" volumeName="kubernetes.io/configmap/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-cabundle" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119643    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be95b15-43a7-4f7e-85c7-14a0adccb875" volumeName="kubernetes.io/secret/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119684    4536 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119657    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06a83215-f446-480d-9036-69d4ac88e172" volumeName="kubernetes.io/projected/06a83215-f446-480d-9036-69d4ac88e172-kube-api-access-8sdz9" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119715    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ac4f3c-9331-4223-bad5-dd4299321eb6" volumeName="kubernetes.io/projected/94ac4f3c-9331-4223-bad5-dd4299321eb6-kube-api-access-fbn98" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119728    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b801f58e-2e25-4e89-b099-030d98aa44c9" volumeName="kubernetes.io/projected/b801f58e-2e25-4e89-b099-030d98aa44c9-kube-api-access-76xfx" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119747    4536 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be95b15-43a7-4f7e-85c7-14a0adccb875" volumeName="kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-config" seLinuxMountContext=""
Oct 30 21:54:29.119744 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119760    4536 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:54:29.121080 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.119767    4536 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:54:29.128099 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.128047    4536 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:54:29.157192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.157158    4536 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:54:29.157192 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.157183    4536 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:54:29.157357 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.157212    4536 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:54:29.157357 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.157252    4536 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:54:29.214418 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.214315    4536 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-138-166.us-west-1.compute.internal\" not found"
Oct 30 21:54:29.232414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.224954    4536 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="293c884dc490fd09f5c91b7a30988dd4dffc4f185c0d222e414052410152adf2" exitCode=0
Oct 30 21:54:29.232414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.224984    4536 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="bc4b6e12db1c6ef185a7a25ec3581e0d0461a6096f09f55e4c2998c9fdf6900e" exitCode=0
Oct 30 21:54:29.232414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.224998    4536 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="973dd411af0169002ab3952f7d8de082d0cb4547727c2f663d0b48dd2dabf48a" exitCode=0
Oct 30 21:54:29.232414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.225007    4536 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="783cdb154bcf555fa439b57843d139d0071d55f67d7d61fa2d197c41284c59ab" exitCode=0
Oct 30 21:54:29.232414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.225019    4536 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="7b09d42c1320dda70f761e9cef2b873c5111d8eb38bf478461c055ccdd33ee19" exitCode=0
Oct 30 21:54:29.232414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.225027    4536 generic.go:356] "Generic (PLEG): container finished" podID="29c88d96-2e8f-4431-9975-f4e8ecba95ba" containerID="978cc22ff4f9d153b69e5eb4ec49f7444b9e34326879095cb90d023c8bf3df75" exitCode=0
Oct 30 21:54:29.245417 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.237090    4536 generic.go:356] "Generic (PLEG): container finished" podID="3f411b386155968c36dc88fdca50807e" containerID="352330a7f52e422b574b5551fe6062c83c43b1b93feab10e7675630e3ebd3bdb" exitCode=0
Oct 30 21:54:29.245417 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.244105    4536 manager.go:324] Recovery completed
Oct 30 21:54:29.257819 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.257796    4536 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:54:29.260480 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.260466    4536 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:54:29.261730 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.261704    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:54:29.261827 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.261745    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:54:29.261827 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.261760    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:54:29.263272 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263245    4536 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:54:29.263272 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263262    4536 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:54:29.263373 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263262    4536 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:54:29.263373 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263360    4536 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:54:29.263538 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263524    4536 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:54:29.263579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263538    4536 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:54:29.263579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263554    4536 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:54:29.263636 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263584    4536 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:54:29.263636 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263590    4536 policy_none.go:50] "Start"
Oct 30 21:54:29.263636 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263600    4536 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:54:29.263636 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263610    4536 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:54:29.263795 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263785    4536 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:54:29.263828 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263796    4536 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:54:29.263828 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.263802    4536 policy_none.go:44] "Start"
Oct 30 21:54:29.264291 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.264276    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:54:29.264345 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.264303    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:54:29.264345 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.264317    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:54:29.269216 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.269169    4536 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:54:29.272436 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.272420    4536 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:54:29.272540 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.272458    4536 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:54:29.272540 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.272470    4536 server.go:87] "Starting device plugin registration server"
Oct 30 21:54:29.272770 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.272757    4536 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:54:29.272822 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.272775    4536 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:54:29.273425 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.273388    4536 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:54:29.273527 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.273497    4536 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:54:29.273581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.273536    4536 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:54:29.273581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.273543    4536 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:54:29.273686 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.273611    4536 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:54:29.273526198Z","UUID":"99871ee9-8508-4f50-8f3f-a9432d67c5a5","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:54:29.276064 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.276049    4536 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:54:29.276182 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.276173    4536 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:54:29.276461 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.276437    4536 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:54:29.276553 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.276477    4536 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-166.us-west-1.compute.internal\" not found"
Oct 30 21:54:29.315544 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.315519    4536 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:54:29.315648 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.315586    4536 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:29.318631 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.318602    4536 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:29.364188 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.364168    4536 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:29.369202 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.369181    4536 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:29.373253 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.373238    4536 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:54:29.374277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.374262    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:54:29.374383 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.374295    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:54:29.374383 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.374310    4536 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-138-166.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:54:29.374383 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.374353    4536 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.458378 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.458327    4536 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"]
Oct 30 21:54:29.458897 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.458865    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"c91ce82603c0eba2cf2b8be390abf9d9","Type":"ContainerStarted","Data":"4ac5084a3009b08ecda012640c5e8032e051c56a645275706882f845d57e6e74"}
Oct 30 21:54:29.458983 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.458904    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"c91ce82603c0eba2cf2b8be390abf9d9","Type":"ContainerStarted","Data":"e3660f083c15cce3bb5eb77d8ddc1e4791402cdac10d59a751b6de51585006fd"}
Oct 30 21:54:29.458983 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.458951    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"3f411b386155968c36dc88fdca50807e","Type":"ContainerStarted","Data":"7e30097d147c7e7c12d542c3c30dc4ff592776576e6598de9c1aa7f519f14d0d"}
Oct 30 21:54:29.458983 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.458972    4536 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.459151 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.459088    4536 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.459151 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.458976    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"3f411b386155968c36dc88fdca50807e","Type":"ContainerDied","Data":"352330a7f52e422b574b5551fe6062c83c43b1b93feab10e7675630e3ebd3bdb"}
Oct 30 21:54:29.459151 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.459132    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal" event={"ID":"3f411b386155968c36dc88fdca50807e","Type":"ContainerStarted","Data":"d749caff3d0d219e203f4762f6437e8fc2b731e668d049d94ea6afbb43db5232"}
Oct 30 21:54:29.490201 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.490177    4536 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.490304 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.490239    4536 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.531725 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.531701    4536 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:29.531803 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.531741    4536 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.536505 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.536490    4536 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:29.536555 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:29.536518    4536 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.617077 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.617050    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.617077 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.617075    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.617242 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.617093    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c91ce82603c0eba2cf2b8be390abf9d9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"c91ce82603c0eba2cf2b8be390abf9d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.717772 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.717694    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.717772 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.717731    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.717772 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.717749    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c91ce82603c0eba2cf2b8be390abf9d9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"c91ce82603c0eba2cf2b8be390abf9d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.717977 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.717790    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c91ce82603c0eba2cf2b8be390abf9d9-config\") pod \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"c91ce82603c0eba2cf2b8be390abf9d9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.717977 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.717798    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:29.717977 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:29.717816    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f411b386155968c36dc88fdca50807e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" (UID: \"3f411b386155968c36dc88fdca50807e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:30.100382 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.100309    4536 apiserver.go:52] "Watching apiserver"
Oct 30 21:54:30.239991 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.239965    4536 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:30.240129 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.240089    4536 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:30.263140 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.263118    4536 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:54:30.263604 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.263579    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-fq4cx","openshift-insights/periodic-gathering-rk2x9-n9c9p","openshift-multus/multus-additional-cni-plugins-gd8pt","openshift-multus/network-metrics-daemon-k4dcq","openshift-image-registry/node-ca-7x2zz","kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal","openshift-dns/dns-default-vx6lh","openshift-cluster-node-tuning-operator/tuned-86pjd","openshift-multus/multus-fpbqf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9","openshift-dns/node-resolver-2r6hv","kube-system/global-pull-secret-syncer-2xm8j","kube-system/konnectivity-agent-pjnv8","openshift-insights/insights-runtime-extractor-r28cn","openshift-network-diagnostics/network-check-target-g96mv","openshift-ovn-kubernetes/ovnkube-node-pk67r","openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99","openshift-ingress-canary/ingress-canary-6w2gg","openshift-service-ca/service-ca-5fffc97bdf-rsm8k"]
Oct 30 21:54:30.267930 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.267877    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.271203 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.271182    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.272054 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.272038    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.301483 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.301459    4536 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:30.301581 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:30.301512    4536 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:30.303256 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.303240    4536 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:30.303352 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:30.303284    4536 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-166.us-west-1.compute.internal"
Oct 30 21:54:30.318931 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.318906    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.318931 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.318908    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.319113 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.319098    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:54:30.319173 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.319161    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.319217 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.319165    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:54:30.319217 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.319183    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.320629 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320602    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:54:30.320629 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320626    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:54:30.320782 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320683    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.320782 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320645    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s9wtv\""
Oct 30 21:54:30.320782 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320652    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.320782 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320780    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ghldd\""
Oct 30 21:54:30.321041 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.320887    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:54:30.321227 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.321211    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:54:30.321286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.321262    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pgbrx\""
Oct 30 21:54:30.321483 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.321468    4536 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:54:30.322612 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322596    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:54:30.322717 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322697    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4dnxg\""
Oct 30 21:54:30.322819 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322804    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z7lww\""
Oct 30 21:54:30.322876 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322833    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-66njg\""
Oct 30 21:54:30.322876 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322833    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8jkh4\""
Oct 30 21:54:30.322876 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322860    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-fd64m\""
Oct 30 21:54:30.322876 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.322874    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t4xfw\""
Oct 30 21:54:30.323324 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.323311    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:54:30.323378 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.323327    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-c7qtj\""
Oct 30 21:54:30.323965 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.323952    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hbp5d\""
Oct 30 21:54:30.324281 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324212    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:54:30.324281 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324242    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:54:30.324281 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324248    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.324281 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324278    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324291    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324300    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324332    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324423    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324428    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324432    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324461    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324463    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:54:30.324559 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.324476    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.340164 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.340148    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.420623 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420557    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46124db8-6b86-4a74-8ef7-80a2a04a8463-konnectivity-ca\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:30.420623 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420598    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-ovn\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.420801 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420625    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-node-log\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.420801 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420652    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-cni-binary-copy\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.420801 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420712    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-multus\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.420920 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420828    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.420920 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420882    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.421003 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420934    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834c4daa-eb50-46b5-919f-c201d9458e11-host\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.421003 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420956    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:54:30.421003 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.420974    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.421003 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421000    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:54:30.421196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421060    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.421196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421087    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-cni-binary-copy\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.421196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421111    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/834c4daa-eb50-46b5-919f-c201d9458e11-serviceca\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.421196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421138    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-conf\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.421388 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421227    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 21:54:30.421388 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421256    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-host\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.421388 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421292    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.421388 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421331    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:54:30.421388 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421370    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/834c4daa-eb50-46b5-919f-c201d9458e11-serviceca\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421431    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46124db8-6b86-4a74-8ef7-80a2a04a8463-agent-certs\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421515    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795f6090-cc3d-4e29-af84-b1de647a037c-tmp-dir\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421554    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ade6beae-e443-46bf-a7fd-3ed608cdd83f-crio-socket\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421580    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-system-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421589    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-plmg6\""
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421612    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-run\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421628    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.421654 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421632    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/795f6090-cc3d-4e29-af84-b1de647a037c-tmp-dir\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421658    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421745    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-netd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421823    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-script-lib\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421886    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hfjcw\""
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421897    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-kubelet\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421938    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.422016 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421936    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-hostroot\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.422313 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422260    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdz9\" (UniqueName: \"kubernetes.io/projected/06a83215-f446-480d-9036-69d4ac88e172-kube-api-access-8sdz9\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.422358 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422312    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:54:30.422358 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422328    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-cert\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:30.422478 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422365    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-modprobe-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.422478 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422419    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-socket-dir-parent\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.422478 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422437    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.422478 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422463    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:54:30.422646 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.421910    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:54:30.422646 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422538    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cnibin\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.422646 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422583    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jwqz7\""
Oct 30 21:54:30.422646 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422586    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.422646 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422625    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysconfig\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.422856 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422655    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.422856 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.422065    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-sqc6l\""
Oct 30 21:54:30.423059 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423041    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 21:54:30.423118 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423054    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-cert\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:30.423118 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423108    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.423259 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423245    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.423435 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423421    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 21:54:30.423521 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423504    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vjl7c\""
Oct 30 21:54:30.423578 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423535    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-j4tgb\""
Oct 30 21:54:30.423624 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423576    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:54:30.423703 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423689    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:54:30.423896 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423878    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-script-lib\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.423988 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423971    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-netns\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.424037 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424007    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.424096 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424055    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.424096 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424069    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5khdr\""
Oct 30 21:54:30.424181 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424109    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-kube-api-access-zwkrc\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.424181 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424146    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade6beae-e443-46bf-a7fd-3ed608cdd83f-data-volume\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.424265 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424197    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-etc-kubernetes\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.424369 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.423981    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:54:30.424438 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424378    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjd6n\" (UniqueName: \"kubernetes.io/projected/834c4daa-eb50-46b5-919f-c201d9458e11-kube-api-access-cjd6n\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.424500 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424438    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cebe5fd4-d933-4a1b-a147-e7c013444b0d-hosts-file\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.424500 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424475    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade6beae-e443-46bf-a7fd-3ed608cdd83f-data-volume\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.424500 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424479    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-systemd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.424686 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424668    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.424772 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424759    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-log-socket\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.424894 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424873    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-k8s-cni-cncf-io\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.425003 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.424988    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-kubelet\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.425096 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425083    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-iptables-alerter-script\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.425196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425184    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbn98\" (UniqueName: \"kubernetes.io/projected/94ac4f3c-9331-4223-bad5-dd4299321eb6-kube-api-access-fbn98\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425217    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425451    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-cabundle\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425502    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btcvc\" (UniqueName: \"kubernetes.io/projected/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-kube-api-access-btcvc\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425537    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425567    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425596    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-iptables-alerter-script\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425604    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425644    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-var-lib-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425677    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-lib-modules\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425707    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425713    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-cabundle\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425740    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425803    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-bin\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.425942    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-env-overrides\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426016    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89lw7\" (UniqueName: \"kubernetes.io/projected/2be95b15-43a7-4f7e-85c7-14a0adccb875-kube-api-access-89lw7\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426068    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-tmp\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426110    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-dbus\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426124    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-env-overrides\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426156    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cebe5fd4-d933-4a1b-a147-e7c013444b0d-tmp-dir\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426187    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-etc-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426241    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426275    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426322    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-socket-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426350    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-device-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426511    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cebe5fd4-d933-4a1b-a147-e7c013444b0d-tmp-dir\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426528    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.426574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426558    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-kubelet-config\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426655    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-bin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426694    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-tmp\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426747    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-conf-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426804    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-multus-certs\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426852    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-registration-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426888    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-sys-fs\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426940    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tljcv\" (UniqueName: \"kubernetes.io/projected/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kube-api-access-tljcv\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.426988    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427021    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmk4b\" (UniqueName: \"kubernetes.io/projected/cebe5fd4-d933-4a1b-a147-e7c013444b0d-kube-api-access-cmk4b\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427074    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-key\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427223    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6090-cc3d-4e29-af84-b1de647a037c-config-volume\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427291    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427361    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph5cp\" (UniqueName: \"kubernetes.io/projected/795f6090-cc3d-4e29-af84-b1de647a037c-kube-api-access-ph5cp\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427442    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/795f6090-cc3d-4e29-af84-b1de647a037c-config-volume\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427441    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qvwd\" (UniqueName: \"kubernetes.io/projected/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-api-access-7qvwd\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.427647 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427525    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-netns\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427913    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-signing-key\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.427981    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428063    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428122    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-var-lib-kubelet\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428165    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428190    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xfx\" (UniqueName: \"kubernetes.io/projected/b801f58e-2e25-4e89-b099-030d98aa44c9-kube-api-access-76xfx\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.428294 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428283    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428318    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-config\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428344    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8sdg\" (UniqueName: \"kubernetes.io/projected/d105096e-ef93-47c0-aae7-41e9b2332753-kube-api-access-h8sdg\") pod \"migrator-66df5d496b-cmf99\" (UID: \"d105096e-ef93-47c0-aae7-41e9b2332753\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428503    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58xfc\" (UniqueName: \"kubernetes.io/projected/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-kube-api-access-58xfc\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428530    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-original-pull-secret\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428546    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-systemd\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428575    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovnkube-config\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428583    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-tuned\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428611    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rm6\" (UniqueName: \"kubernetes.io/projected/29c88d96-2e8f-4431-9975-f4e8ecba95ba-kube-api-access-j2rm6\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428636    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-systemd-units\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428664    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-cnibin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.428690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428677    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428784    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-tuned\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428823    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-system-cni-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428846    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-slash\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428888    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-host-slash\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428925    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-sys\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428949    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.428982    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.429034    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovn-node-metrics-cert\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.429092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.429079    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-multus-daemon-config\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.429339 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.429118    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-kubernetes\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.429339 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.429166    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2be95b15-43a7-4f7e-85c7-14a0adccb875-ovn-node-metrics-cert\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.429416 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.429351    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06a83215-f446-480d-9036-69d4ac88e172-multus-daemon-config\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.429541 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.429525    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.431342 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.431328    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46124db8-6b86-4a74-8ef7-80a2a04a8463-konnectivity-ca\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:30.432557 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.432543    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46124db8-6b86-4a74-8ef7-80a2a04a8463-agent-certs\") pod \"konnectivity-agent-pjnv8\" (UID: \"46124db8-6b86-4a74-8ef7-80a2a04a8463\") " pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:30.484050 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.484027    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") pod \"periodic-gathering-rk2x9-n9c9p\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") " pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:54:30.487838 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.487818    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkrc\" (UniqueName: \"kubernetes.io/projected/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-kube-api-access-zwkrc\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.487973 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.487953    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjd6n\" (UniqueName: \"kubernetes.io/projected/834c4daa-eb50-46b5-919f-c201d9458e11-kube-api-access-cjd6n\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.488044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.488030    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbn98\" (UniqueName: \"kubernetes.io/projected/94ac4f3c-9331-4223-bad5-dd4299321eb6-kube-api-access-fbn98\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.489272 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.489255    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph5cp\" (UniqueName: \"kubernetes.io/projected/795f6090-cc3d-4e29-af84-b1de647a037c-kube-api-access-ph5cp\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.489457 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.489441    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tljcv\" (UniqueName: \"kubernetes.io/projected/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kube-api-access-tljcv\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.489577 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.489547    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmk4b\" (UniqueName: \"kubernetes.io/projected/cebe5fd4-d933-4a1b-a147-e7c013444b0d-kube-api-access-cmk4b\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.489848 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.489828    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95k2j\" (UniqueName: \"kubernetes.io/projected/eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1-kube-api-access-95k2j\") pod \"network-check-target-g96mv\" (UID: \"eda1d366-0cbe-4eea-a3c3-7c08b2ff04c1\") " pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:30.490001 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.489986    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcvc\" (UniqueName: \"kubernetes.io/projected/e4db1897-c6cc-4c76-810a-e7ed9e00d15e-kube-api-access-btcvc\") pod \"service-ca-5fffc97bdf-rsm8k\" (UID: \"e4db1897-c6cc-4c76-810a-e7ed9e00d15e\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rsm8k"
Oct 30 21:54:30.490414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.490375    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sdz9\" (UniqueName: \"kubernetes.io/projected/06a83215-f446-480d-9036-69d4ac88e172-kube-api-access-8sdz9\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.490803 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.490780    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89lw7\" (UniqueName: \"kubernetes.io/projected/2be95b15-43a7-4f7e-85c7-14a0adccb875-kube-api-access-89lw7\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.495752 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.495729    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8sdg\" (UniqueName: \"kubernetes.io/projected/d105096e-ef93-47c0-aae7-41e9b2332753-kube-api-access-h8sdg\") pod \"migrator-66df5d496b-cmf99\" (UID: \"d105096e-ef93-47c0-aae7-41e9b2332753\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-cmf99"
Oct 30 21:54:30.495843 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.495809    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qvwd\" (UniqueName: \"kubernetes.io/projected/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-api-access-7qvwd\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.495965 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.495946    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xfx\" (UniqueName: \"kubernetes.io/projected/b801f58e-2e25-4e89-b099-030d98aa44c9-kube-api-access-76xfx\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.496017 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.495937    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rm6\" (UniqueName: \"kubernetes.io/projected/29c88d96-2e8f-4431-9975-f4e8ecba95ba-kube-api-access-j2rm6\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.496059 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.496043    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58xfc\" (UniqueName: \"kubernetes.io/projected/684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27-kube-api-access-58xfc\") pod \"ingress-canary-6w2gg\" (UID: \"684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27\") " pod="openshift-ingress-canary/ingress-canary-6w2gg"
Oct 30 21:54:30.529998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.529976    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cebe5fd4-d933-4a1b-a147-e7c013444b0d-hosts-file\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.530064 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530009    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-systemd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530108 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530091    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cebe5fd4-d933-4a1b-a147-e7c013444b0d-hosts-file\") pod \"node-resolver-2r6hv\" (UID: \"cebe5fd4-d933-4a1b-a147-e7c013444b0d\") " pod="openshift-dns/node-resolver-2r6hv"
Oct 30 21:54:30.530155 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530089    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530155 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530127    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530155 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530134    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-systemd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530166    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-log-socket\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530198    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-k8s-cni-cncf-io\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530221    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-kubelet\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530239    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-log-socket\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530249    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530258    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-k8s-cni-cncf-io\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530271    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-kubelet\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530293 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530274    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530300    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530323    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-var-lib-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530347    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-lib-modules\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530371    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530411    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-bin\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530445    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-var-lib-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:30.530462    4536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530492    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-dbus\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530492    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530520    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-etc-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530467    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-bin\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:30.530554    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:31.030533522 +0000 UTC m=+2.018341796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530558    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-dbus\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530548    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-lib-modules\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530571    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-etc-openvswitch\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530577    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530604    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-socket-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530620    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-device-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530635    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-kubelet-config\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530624    4536 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:54:30.530668 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530661    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-bin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530684    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-conf-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530690    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-socket-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530694    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-device-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530709    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-multus-certs\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530731    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-bin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530734    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-registration-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530707    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d64e1a7d-50f4-4f15-ab5c-71921ae2af7e-kubelet-config\") pod \"global-pull-secret-syncer-2xm8j\" (UID: \"d64e1a7d-50f4-4f15-ab5c-71921ae2af7e\") " pod="kube-system/global-pull-secret-syncer-2xm8j"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530738    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-conf-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530762    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-multus-certs\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530774    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-registration-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530771    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-sys-fs\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530814    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-netns\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530638    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530838    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530874    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-run-netns\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530881    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-sys-fs\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530889    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530913    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-var-lib-kubelet\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530928    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530941    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-systemd\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530966    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-systemd-units\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530975    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-var-lib-kubelet\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.530992    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-cnibin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531015    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-system-cni-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531020    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531023    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-systemd\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531038    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-slash\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531063    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-systemd-units\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531081    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-host-slash\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531110    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-sys\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531114    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-cnibin\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531135    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531161    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95-host-slash\") pod \"iptables-alerter-fq4cx\" (UID: \"e9dbad13-f5f2-4f7f-9c68-2cdab73a8f95\") " pod="openshift-network-operator/iptables-alerter-fq4cx"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531163    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-kubernetes\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531216    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-ovn\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531260    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-node-log\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531276    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-system-cni-dir\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531285    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-multus\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531309    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834c4daa-eb50-46b5-919f-c201d9458e11-host\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531274    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-slash\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531367    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-sys\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531332    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-conf\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531327    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-kubernetes\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531382    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-node-log\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531431    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-host\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531441    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-run-ovn\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531458    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-cni-multus\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531459    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ade6beae-e443-46bf-a7fd-3ed608cdd83f-crio-socket\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531473    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysctl-conf\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531496    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531499    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ade6beae-e443-46bf-a7fd-3ed608cdd83f-crio-socket\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531530    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-system-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531559    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-run\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531578    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/834c4daa-eb50-46b5-919f-c201d9458e11-host\") pod \"node-ca-7x2zz\" (UID: \"834c4daa-eb50-46b5-919f-c201d9458e11\") " pod="openshift-image-registry/node-ca-7x2zz"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531584    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-netd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.531581 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531610    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-kubelet\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531633    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-system-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531640    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-run\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531637    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-hostroot\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531677    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-cni-netd\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531682    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-host\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531716    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-host-var-lib-kubelet\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531738    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-hostroot\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531778    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-modprobe-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531823    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-socket-dir-parent\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531877    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cnibin\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531882    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-modprobe-d\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531891    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-socket-dir-parent\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531921    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysconfig\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531921    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29c88d96-2e8f-4431-9975-f4e8ecba95ba-cnibin\") pod \"multus-additional-cni-plugins-gd8pt\" (UID: \"29c88d96-2e8f-4431-9975-f4e8ecba95ba\") " pod="openshift-multus/multus-additional-cni-plugins-gd8pt"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531947    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531970    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-netns\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531976    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94ac4f3c-9331-4223-bad5-dd4299321eb6-etc-sysconfig\") pod \"tuned-86pjd\" (UID: \"94ac4f3c-9331-4223-bad5-dd4299321eb6\") " pod="openshift-cluster-node-tuning-operator/tuned-86pjd"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.531994    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:30.532026    4536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.532026    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2be95b15-43a7-4f7e-85c7-14a0adccb875-host-run-netns\") pod \"ovnkube-node-pk67r\" (UID: \"2be95b15-43a7-4f7e-85c7-14a0adccb875\") " pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.532050    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-multus-cni-dir\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.532027    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.532073    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80c062cd-a506-47bf-ad60-4bcfe3c5a19e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gb4g9\" (UID: \"80c062cd-a506-47bf-ad60-4bcfe3c5a19e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gb4g9"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.532089    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-etc-kubernetes\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.532110    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06a83215-f446-480d-9036-69d4ac88e172-etc-kubernetes\") pod \"multus-fpbqf\" (UID: \"06a83215-f446-480d-9036-69d4ac88e172\") " pod="openshift-multus/multus-fpbqf"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:30.532137    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:31.032119207 +0000 UTC m=+2.019927455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.533057    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b801f58e-2e25-4e89-b099-030d98aa44c9-metrics-certs\") pod \"network-metrics-daemon-k4dcq\" (UID: \"b801f58e-2e25-4e89-b099-030d98aa44c9\") " pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.533337 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.533093    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/795f6090-cc3d-4e29-af84-b1de647a037c-metrics-tls\") pod \"dns-default-vx6lh\" (UID: \"795f6090-cc3d-4e29-af84-b1de647a037c\") " pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.568545 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.568529    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4dcq"
Oct 30 21:54:30.572424 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.572385    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:30.650753 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.650716    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6w2gg_684ddbe7-01fe-42cc-a2c2-ffb9b3cd1e27/serve-healthcheck-canary/0.log"
Oct 30 21:54:30.997782 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:30.997747    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k4dcq"]
Oct 30 21:54:31.001065 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:31.001037    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb801f58e_2e25_4e89_b099_030d98aa44c9.slice/crio-d12dd8308c9326d5b1fa839a432fd969ebd4587b2db05cf68c1298ba35c1100e WatchSource:0}: Error finding container d12dd8308c9326d5b1fa839a432fd969ebd4587b2db05cf68c1298ba35c1100e: Status 404 returned error can't find the container with id d12dd8308c9326d5b1fa839a432fd969ebd4587b2db05cf68c1298ba35c1100e
Oct 30 21:54:31.002991 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.002977    4536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:54:31.035384 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.035357    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:31.035499 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.035411    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:31.035499 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:31.035479    4536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:31.035576 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:31.035518    4536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:31.035576 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:31.035558    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:32.03553976 +0000 UTC m=+3.023348014 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:31.035641 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:31.035586    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:32.035566918 +0000 UTC m=+3.023375160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:31.051574 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.051551    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vx6lh"]
Oct 30 21:54:31.052003 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.051990    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-cmf99_d105096e-ef93-47c0-aae7-41e9b2332753/migrator/0.log"
Oct 30 21:54:31.054518 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:31.054498    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795f6090_cc3d_4e29_af84_b1de647a037c.slice/crio-4a721993dafc56c28a5d35d8829b7d43ea511177f9a1af926c5188d9ef26bf40 WatchSource:0}: Error finding container 4a721993dafc56c28a5d35d8829b7d43ea511177f9a1af926c5188d9ef26bf40: Status 404 returned error can't find the container with id 4a721993dafc56c28a5d35d8829b7d43ea511177f9a1af926c5188d9ef26bf40
Oct 30 21:54:31.246685 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.246655    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k4dcq" event={"ID":"b801f58e-2e25-4e89-b099-030d98aa44c9","Type":"ContainerStarted","Data":"d12dd8308c9326d5b1fa839a432fd969ebd4587b2db05cf68c1298ba35c1100e"}
Oct 30 21:54:31.248074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:31.248009    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vx6lh" event={"ID":"795f6090-cc3d-4e29-af84-b1de647a037c","Type":"ContainerStarted","Data":"4a721993dafc56c28a5d35d8829b7d43ea511177f9a1af926c5188d9ef26bf40"}
Oct 30 21:54:32.044499 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:32.044289    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:32.044499 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:32.044391    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:32.044499 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:32.044488    4536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:32.044788 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:32.044562    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:34.044546773 +0000 UTC m=+5.032355014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:32.044788 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:32.044488    4536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:54:32.044788 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:32.044635    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:34.044621761 +0000 UTC m=+5.032430020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : configmap "kube-rbac-proxy" not found
Oct 30 21:54:33.260756 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:33.260724    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k4dcq" event={"ID":"b801f58e-2e25-4e89-b099-030d98aa44c9","Type":"ContainerStarted","Data":"eef766101d268781ea2f062efd4be439a253ea4c2351adb313911cc92de3cc1a"}
Oct 30 21:54:33.261167 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:33.260776    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k4dcq" event={"ID":"b801f58e-2e25-4e89-b099-030d98aa44c9","Type":"ContainerStarted","Data":"aac7a22ff7cbd617eaa694c1c5a8ba29cb4e70a809d100a6a44a2d2668983bef"}
Oct 30 21:54:34.058816 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:34.058782    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:34.059045 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:34.058837    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:34.059045 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:34.058951    4536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:34.059045 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:34.059032    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:38.059012572 +0000 UTC m=+9.046820827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:34.059340 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:34.059320    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ade6beae-e443-46bf-a7fd-3ed608cdd83f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:34.265713 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:34.265679    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vx6lh" event={"ID":"795f6090-cc3d-4e29-af84-b1de647a037c","Type":"ContainerStarted","Data":"c9e587c33425fdccc48fadcc9eaf9ddc3add53ed7c220c43566afac3040cbe25"}
Oct 30 21:54:34.266103 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:34.265733    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vx6lh" event={"ID":"795f6090-cc3d-4e29-af84-b1de647a037c","Type":"ContainerStarted","Data":"a455d07d7feee0c190bdad4c4a6d79b143eb75fbeeb4d0ac5be95e74bf27ccba"}
Oct 30 21:54:34.266103 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:34.265897    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:37.368266 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:37.368238    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:38.085446 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:38.085413    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:38.085656 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:38.085553    4536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:38.085656 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:38.085628    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls podName:ade6beae-e443-46bf-a7fd-3ed608cdd83f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:46.085607059 +0000 UTC m=+17.073415313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r28cn" (UID: "ade6beae-e443-46bf-a7fd-3ed608cdd83f") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:54:38.242341 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:38.242312    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:54:38.278666 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:38.278641    4536 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:54:38.673202 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:38.673164    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:44.272157 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:44.272118    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vx6lh"
Oct 30 21:54:46.140851 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:46.140810    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:46.143315 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:46.143289    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ade6beae-e443-46bf-a7fd-3ed608cdd83f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r28cn\" (UID: \"ade6beae-e443-46bf-a7fd-3ed608cdd83f\") " pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:46.173328 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:46.173287    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r28cn"
Oct 30 21:54:46.546748 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:46.546713    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r28cn"]
Oct 30 21:54:46.549964 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:54:46.549923    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade6beae_e443_46bf_a7fd_3ed608cdd83f.slice/crio-9b42c866079b73cd19483d3e3915fd86fdc961907692cb8281d687800c03e301 WatchSource:0}: Error finding container 9b42c866079b73cd19483d3e3915fd86fdc961907692cb8281d687800c03e301: Status 404 returned error can't find the container with id 9b42c866079b73cd19483d3e3915fd86fdc961907692cb8281d687800c03e301
Oct 30 21:54:47.304499 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:47.304460    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r28cn" event={"ID":"ade6beae-e443-46bf-a7fd-3ed608cdd83f","Type":"ContainerStarted","Data":"cffee33ab51857125cb0844e92b3743c83527356d3326f151e550fc7e86cddfe"}
Oct 30 21:54:47.304920 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:47.304523    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r28cn" event={"ID":"ade6beae-e443-46bf-a7fd-3ed608cdd83f","Type":"ContainerStarted","Data":"9b42c866079b73cd19483d3e3915fd86fdc961907692cb8281d687800c03e301"}
Oct 30 21:54:48.309240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:48.309212    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r28cn" event={"ID":"ade6beae-e443-46bf-a7fd-3ed608cdd83f","Type":"ContainerStarted","Data":"1dcd3ddeb787911b79ae4f4c0ab5fb4037cf9b06fe3227781979b3891a2951a3"}
Oct 30 21:54:50.316365 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:50.316323    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r28cn" event={"ID":"ade6beae-e443-46bf-a7fd-3ed608cdd83f","Type":"ContainerStarted","Data":"c49203bcd40bec278fd0d940dbf3dcd1006b2cdfa0439c5fe045eb76d0f68fa2"}
Oct 30 21:54:50.803480 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:54:50.803445    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:54:53.548278 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:53.548249    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pjnv8"
Oct 30 21:54:58.676097 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:58.676062    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g96mv"
Oct 30 21:54:59.006008 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:54:59.005981    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pk67r"
Oct 30 21:55:03.591987 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.591952    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"]
Oct 30 21:55:03.626528 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.626419    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.656762 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.656728    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"]
Oct 30 21:55:03.705952 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.705902    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:55:03.705952 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.705941    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:55:03.706185 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.706009    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:55:03.730026 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.729997    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7hbfw"]
Oct 30 21:55:03.733381 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.733363    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:55:03.744514 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.744491    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.755594 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.755570    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:55:03.755701 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.755616    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zhdhf\""
Oct 30 21:55:03.757855 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.757836    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fcda885e-673f-4be4-bf9e-7a93d667ed5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.757924 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.757864    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p29r\" (UniqueName: \"kubernetes.io/projected/fcda885e-673f-4be4-bf9e-7a93d667ed5b-kube-api-access-9p29r\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.757924 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.757897    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcda885e-673f-4be4-bf9e-7a93d667ed5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.758035 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.758014    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcda885e-673f-4be4-bf9e-7a93d667ed5b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.829250 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.829214    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-szz58\""
Oct 30 21:55:03.829250 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.829229    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:55:03.829477 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.829260    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:55:03.858633 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858557    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fcda885e-673f-4be4-bf9e-7a93d667ed5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.858633 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858602    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39b94723-b921-47ff-96f9-f2a9b948f92c-metrics-client-ca\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.858633 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858629    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-sys\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.858867 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858655    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.858867 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858687    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkz4r\" (UniqueName: \"kubernetes.io/projected/39b94723-b921-47ff-96f9-f2a9b948f92c-kube-api-access-fkz4r\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.858867 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858744    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-wtmp\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.858867 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858789    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcda885e-673f-4be4-bf9e-7a93d667ed5b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.859071 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858882    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-textfile\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.859071 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858909    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p29r\" (UniqueName: \"kubernetes.io/projected/fcda885e-673f-4be4-bf9e-7a93d667ed5b-kube-api-access-9p29r\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.859071 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858927    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-tls\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.859071 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858945    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-root\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.859071 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858969    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcda885e-673f-4be4-bf9e-7a93d667ed5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.859071 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.858984    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-accelerators-collector-config\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.859527 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.859507    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fcda885e-673f-4be4-bf9e-7a93d667ed5b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.861279 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.861263    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fcda885e-673f-4be4-bf9e-7a93d667ed5b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.861376 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.861359    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fcda885e-673f-4be4-bf9e-7a93d667ed5b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.880371 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.880345    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:55:03.921354 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.921321    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p29r\" (UniqueName: \"kubernetes.io/projected/fcda885e-673f-4be4-bf9e-7a93d667ed5b-kube-api-access-9p29r\") pod \"openshift-state-metrics-7bd7489957-f5f7m\" (UID: \"fcda885e-673f-4be4-bf9e-7a93d667ed5b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:03.959587 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959548    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkz4r\" (UniqueName: \"kubernetes.io/projected/39b94723-b921-47ff-96f9-f2a9b948f92c-kube-api-access-fkz4r\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.959587 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959589    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-wtmp\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.959830 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959614    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-textfile\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.959830 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959740    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-tls\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.959830 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959745    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-wtmp\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.959830 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959778    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-root\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.959830 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959819    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-accelerators-collector-config\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959879    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-root\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959900    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39b94723-b921-47ff-96f9-f2a9b948f92c-metrics-client-ca\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959925    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-sys\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959937    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-textfile\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959958    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960044 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.959997    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39b94723-b921-47ff-96f9-f2a9b948f92c-sys\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960421 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.960380    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-accelerators-collector-config\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.960521 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.960502    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39b94723-b921-47ff-96f9-f2a9b948f92c-metrics-client-ca\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.962076 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.962052    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-tls\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.962229 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.962214    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39b94723-b921-47ff-96f9-f2a9b948f92c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:03.974184 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:03.974157    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"
Oct 30 21:55:04.022933 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:04.022892    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkz4r\" (UniqueName: \"kubernetes.io/projected/39b94723-b921-47ff-96f9-f2a9b948f92c-kube-api-access-fkz4r\") pod \"node-exporter-7hbfw\" (UID: \"39b94723-b921-47ff-96f9-f2a9b948f92c\") " pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:04.093410 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:04.093370    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7hbfw"
Oct 30 21:55:04.129738 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:55:04.129706    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b94723_b921_47ff_96f9_f2a9b948f92c.slice/crio-8571a3bb8d4d3a0aea3ef328561ff4372424f368c16e134995a58ead76ee1721 WatchSource:0}: Error finding container 8571a3bb8d4d3a0aea3ef328561ff4372424f368c16e134995a58ead76ee1721: Status 404 returned error can't find the container with id 8571a3bb8d4d3a0aea3ef328561ff4372424f368c16e134995a58ead76ee1721
Oct 30 21:55:04.358055 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:04.358027    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7hbfw" event={"ID":"39b94723-b921-47ff-96f9-f2a9b948f92c","Type":"ContainerStarted","Data":"8571a3bb8d4d3a0aea3ef328561ff4372424f368c16e134995a58ead76ee1721"}
Oct 30 21:55:04.424150 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:04.424080    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m"]
Oct 30 21:55:04.427102 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:55:04.427073    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcda885e_673f_4be4_bf9e_7a93d667ed5b.slice/crio-b4f3842d298f8c001346febe61ef7851d94b417413a348bbd177ace9c7e71559 WatchSource:0}: Error finding container b4f3842d298f8c001346febe61ef7851d94b417413a348bbd177ace9c7e71559: Status 404 returned error can't find the container with id b4f3842d298f8c001346febe61ef7851d94b417413a348bbd177ace9c7e71559
Oct 30 21:55:05.363042 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.363011    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m" event={"ID":"fcda885e-673f-4be4-bf9e-7a93d667ed5b","Type":"ContainerStarted","Data":"6bd0f9dfe120c79b9a7f886d662147f45c3a0eb72f9c7fb862968be9146b575f"}
Oct 30 21:55:05.363448 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.363065    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m" event={"ID":"fcda885e-673f-4be4-bf9e-7a93d667ed5b","Type":"ContainerStarted","Data":"f3ef33f8507ae8f93c030106ca0d08fa0fb1a33c453994246a8031268038a593"}
Oct 30 21:55:05.363448 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.363084    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m" event={"ID":"fcda885e-673f-4be4-bf9e-7a93d667ed5b","Type":"ContainerStarted","Data":"b4f3842d298f8c001346febe61ef7851d94b417413a348bbd177ace9c7e71559"}
Oct 30 21:55:05.917146 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.917073    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:05.919591 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.919571    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974266    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:55:05.974602 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974452    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:55:05.974602 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974516    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974602 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974549    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974602 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974570    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974602 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974593    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974703    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974725    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974734    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974744    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974759    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974777    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974789    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974836    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974834    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.974877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974846    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:55:05.975277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974901    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5n786\""
Oct 30 21:55:05.975277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.974928    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.975277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.975002    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.975277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.975036    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.975277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.975059    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:05.975277 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:05.975084    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqkn\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.073781 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.073752    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:55:06.074789 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.074766    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075571    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075608    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075633    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075658    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075678    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075702    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075736    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075762    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075784    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075811    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqkn\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075859    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075889    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.076222 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.075920    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.078241 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.078210    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.078879 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.078851    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.079225 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.079202    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.079366 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.079326    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.079366 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.079355    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.081776 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.081751    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.081873 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.081811    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.081873 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.081863    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.082908 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.082888    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.083023 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.083002    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.083059 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.083008    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.083211 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.083189    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.145506 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.141209    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqkn\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn\") pod \"alertmanager-main-0\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.237636 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.237598    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:06.368474 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.368427    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m" event={"ID":"fcda885e-673f-4be4-bf9e-7a93d667ed5b","Type":"ContainerStarted","Data":"29afff82378f22c1edf3274b94184dbfffbfb61bfd1b7112a5175e4924731924"}
Oct 30 21:55:06.370149 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.370123    4536 generic.go:356] "Generic (PLEG): container finished" podID="39b94723-b921-47ff-96f9-f2a9b948f92c" containerID="ed313f126d3751ac6b0351e490f0bfa8f99a391242ae8c3c28d7e46668a94774" exitCode=0
Oct 30 21:55:06.370247 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.370168    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7hbfw" event={"ID":"39b94723-b921-47ff-96f9-f2a9b948f92c","Type":"ContainerDied","Data":"ed313f126d3751ac6b0351e490f0bfa8f99a391242ae8c3c28d7e46668a94774"}
Oct 30 21:55:06.489305 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.488650    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f5f7m" podStartSLOduration=2.0365829 podStartE2EDuration="3.488636843s" totalImagesPullingTime="1.452053943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:06.487719951 +0000 UTC m=+37.475528215" watchObservedRunningTime="2026-10-30 21:55:06.488636843 +0000 UTC m=+37.476445123"
Oct 30 21:55:06.666061 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:06.665525    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:06.668232 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:55:06.668202    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef84e48_84e4_4876_8cac_e4f8eed1af3e.slice/crio-73e6852597a78a4748c26b54ade0082745f996dc7d6e155fa35e2ba2300655dc WatchSource:0}: Error finding container 73e6852597a78a4748c26b54ade0082745f996dc7d6e155fa35e2ba2300655dc: Status 404 returned error can't find the container with id 73e6852597a78a4748c26b54ade0082745f996dc7d6e155fa35e2ba2300655dc
Oct 30 21:55:07.375610 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:07.375576    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7hbfw" event={"ID":"39b94723-b921-47ff-96f9-f2a9b948f92c","Type":"ContainerStarted","Data":"610e4d012127a38bce2d8a94b4ca9a2479bb1b60001a0de39d921fba70889077"}
Oct 30 21:55:07.376053 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:07.375634    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7hbfw" event={"ID":"39b94723-b921-47ff-96f9-f2a9b948f92c","Type":"ContainerStarted","Data":"5eec0dbfb98e149d85296ff085e6e87f15763b0798e2aaa071c0001bb87f4cfd"}
Oct 30 21:55:07.376901 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:07.376875    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"73e6852597a78a4748c26b54ade0082745f996dc7d6e155fa35e2ba2300655dc"}
Oct 30 21:55:07.503341 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:07.503286    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7hbfw" podStartSLOduration=2.943325268 podStartE2EDuration="4.503272398s" totalImagesPullingTime="1.477339095s" totalInitContainerRuntime="82.608035ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:07.502809137 +0000 UTC m=+38.490617413" watchObservedRunningTime="2026-10-30 21:55:07.503272398 +0000 UTC m=+38.491080669"
Oct 30 21:55:08.381899 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.381864    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a" exitCode=0
Oct 30 21:55:08.382291 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.381948    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"}
Oct 30 21:55:08.387555 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.387535    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"]
Oct 30 21:55:08.389754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.389739    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.485918 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.485886    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:55:08.486097 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.485932    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dtf27\""
Oct 30 21:55:08.486612 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.486594    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"]
Oct 30 21:55:08.497413 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.497324    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e5493e9-c44c-4903-bb43-007730d23b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-scd5x\" (UID: \"1e5493e9-c44c-4903-bb43-007730d23b5f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.497589 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.497564    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjgx\" (UniqueName: \"kubernetes.io/projected/1e5493e9-c44c-4903-bb43-007730d23b5f-kube-api-access-npjgx\") pod \"monitoring-plugin-77f4f76594-scd5x\" (UID: \"1e5493e9-c44c-4903-bb43-007730d23b5f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.598450 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.598414    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e5493e9-c44c-4903-bb43-007730d23b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-scd5x\" (UID: \"1e5493e9-c44c-4903-bb43-007730d23b5f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.598450 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.598451    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjgx\" (UniqueName: \"kubernetes.io/projected/1e5493e9-c44c-4903-bb43-007730d23b5f-kube-api-access-npjgx\") pod \"monitoring-plugin-77f4f76594-scd5x\" (UID: \"1e5493e9-c44c-4903-bb43-007730d23b5f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.600997 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.600971    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1e5493e9-c44c-4903-bb43-007730d23b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-scd5x\" (UID: \"1e5493e9-c44c-4903-bb43-007730d23b5f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.661807 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.661774    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjgx\" (UniqueName: \"kubernetes.io/projected/1e5493e9-c44c-4903-bb43-007730d23b5f-kube-api-access-npjgx\") pod \"monitoring-plugin-77f4f76594-scd5x\" (UID: \"1e5493e9-c44c-4903-bb43-007730d23b5f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.685348 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.685316    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d94746fd5-q662f"]
Oct 30 21:55:08.687343 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.687323    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.703711 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.703688    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:08.742762 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.742678    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:55:08.742762 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.742728    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:55:08.742762 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.742738    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:55:08.743011 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.742927    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bb646\""
Oct 30 21:55:08.743011 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.742990    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:55:08.743094 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.743075    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5qm8sgnrnc9kf\""
Oct 30 21:55:08.749589 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.749568    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d94746fd5-q662f"]
Oct 30 21:55:08.800843 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.800814    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c91e5088-18a7-4ae9-8274-0943c3a512be-audit-log\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.801125 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.800853    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkdx7\" (UniqueName: \"kubernetes.io/projected/c91e5088-18a7-4ae9-8274-0943c3a512be-kube-api-access-jkdx7\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.801125 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.800877    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91e5088-18a7-4ae9-8274-0943c3a512be-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.801125 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.800895    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c91e5088-18a7-4ae9-8274-0943c3a512be-metrics-server-audit-profiles\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.801125 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.801059    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-secret-metrics-server-client-certs\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.801125 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.801096    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-secret-metrics-server-tls\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.801335 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.801164    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-client-ca-bundle\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901757 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901719    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c91e5088-18a7-4ae9-8274-0943c3a512be-audit-log\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901757 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901763    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkdx7\" (UniqueName: \"kubernetes.io/projected/c91e5088-18a7-4ae9-8274-0943c3a512be-kube-api-access-jkdx7\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901999 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901786    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91e5088-18a7-4ae9-8274-0943c3a512be-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901999 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901802    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c91e5088-18a7-4ae9-8274-0943c3a512be-metrics-server-audit-profiles\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901999 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901834    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-secret-metrics-server-client-certs\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901999 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901862    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-secret-metrics-server-tls\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.901999 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.901897    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-client-ca-bundle\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.902323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.902182    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c91e5088-18a7-4ae9-8274-0943c3a512be-audit-log\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.902567 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.902542    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c91e5088-18a7-4ae9-8274-0943c3a512be-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.903095 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.903074    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c91e5088-18a7-4ae9-8274-0943c3a512be-metrics-server-audit-profiles\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.904508 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.904482    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-secret-metrics-server-client-certs\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.904598 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.904561    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-secret-metrics-server-tls\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.904652 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.904635    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c91e5088-18a7-4ae9-8274-0943c3a512be-client-ca-bundle\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.964190 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.964160    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkdx7\" (UniqueName: \"kubernetes.io/projected/c91e5088-18a7-4ae9-8274-0943c3a512be-kube-api-access-jkdx7\") pod \"metrics-server-5d94746fd5-q662f\" (UID: \"c91e5088-18a7-4ae9-8274-0943c3a512be\") " pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:08.997122 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:08.997049    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:09.075213 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:09.075188    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"]
Oct 30 21:55:09.077647 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:55:09.077624    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5493e9_c44c_4903_bb43_007730d23b5f.slice/crio-8369131ec5bb2d17e1b6a447a3b3a5e69f2aaa5b54f7ffcb5a30066b2ccf3cac WatchSource:0}: Error finding container 8369131ec5bb2d17e1b6a447a3b3a5e69f2aaa5b54f7ffcb5a30066b2ccf3cac: Status 404 returned error can't find the container with id 8369131ec5bb2d17e1b6a447a3b3a5e69f2aaa5b54f7ffcb5a30066b2ccf3cac
Oct 30 21:55:09.367133 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:09.367103    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d94746fd5-q662f"]
Oct 30 21:55:09.369542 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:55:09.369506    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91e5088_18a7_4ae9_8274_0943c3a512be.slice/crio-861fe8805aa95134d7d6b56564f8aaafe3e92020b6f24714ce0ca82ffb9a9810 WatchSource:0}: Error finding container 861fe8805aa95134d7d6b56564f8aaafe3e92020b6f24714ce0ca82ffb9a9810: Status 404 returned error can't find the container with id 861fe8805aa95134d7d6b56564f8aaafe3e92020b6f24714ce0ca82ffb9a9810
Oct 30 21:55:09.385918 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:09.385892    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f" event={"ID":"c91e5088-18a7-4ae9-8274-0943c3a512be","Type":"ContainerStarted","Data":"861fe8805aa95134d7d6b56564f8aaafe3e92020b6f24714ce0ca82ffb9a9810"}
Oct 30 21:55:09.387029 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:09.387010    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x" event={"ID":"1e5493e9-c44c-4903-bb43-007730d23b5f","Type":"ContainerStarted","Data":"8369131ec5bb2d17e1b6a447a3b3a5e69f2aaa5b54f7ffcb5a30066b2ccf3cac"}
Oct 30 21:55:11.396512 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.396472    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f" event={"ID":"c91e5088-18a7-4ae9-8274-0943c3a512be","Type":"ContainerStarted","Data":"56a813201cc4bcd9d8e40354cadcd558044e09fdecabde8d3df49e0473682f9f"}
Oct 30 21:55:11.398086 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.398054    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x" event={"ID":"1e5493e9-c44c-4903-bb43-007730d23b5f","Type":"ContainerStarted","Data":"ffce63bd922dc2c3832c9de08aecd818d63cf3f98d45db5cc5764c234adb5bc9"}
Oct 30 21:55:11.398217 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.398101    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:11.401562 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.401540    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"}
Oct 30 21:55:11.401686 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.401605    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"}
Oct 30 21:55:11.401686 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.401646    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"}
Oct 30 21:55:11.401686 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.401681    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"}
Oct 30 21:55:11.401806 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.401715    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"}
Oct 30 21:55:11.403589 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.403572    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x"
Oct 30 21:55:11.521251 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.521150    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f" podStartSLOduration=1.637700594 podStartE2EDuration="3.521136326s" totalImagesPullingTime="1.883435732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:11.520770616 +0000 UTC m=+42.508578879" watchObservedRunningTime="2026-10-30 21:55:11.521136326 +0000 UTC m=+42.508944581"
Oct 30 21:55:11.637953 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:11.637895    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-scd5x" podStartSLOduration=1.744017807 podStartE2EDuration="3.637879956s" totalImagesPullingTime="1.893862149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:11.637004933 +0000 UTC m=+42.624813197" watchObservedRunningTime="2026-10-30 21:55:11.637879956 +0000 UTC m=+42.625688233"
Oct 30 21:55:12.408636 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.408544    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerStarted","Data":"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"}
Oct 30 21:55:12.699609 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.699553    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.891613926 podStartE2EDuration="7.699537324s" totalImagesPullingTime="4.797028087s" totalInitContainerRuntime="10.895311ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:12.69880236 +0000 UTC m=+43.686610623" watchObservedRunningTime="2026-10-30 21:55:12.699537324 +0000 UTC m=+43.687345622"
Oct 30 21:55:12.711891 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.711861    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ffcf9bf78-k4pfk"]
Oct 30 21:55:12.713951 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.713935    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.770683 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.770653    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:55:12.770831 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.770815    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:55:12.770932 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.770910    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:55:12.771064 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.771048    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wwkpf\""
Oct 30 21:55:12.771129 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.771101    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:55:12.771295 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.771281    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:55:12.771365 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.771292    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:55:12.771365 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.771336    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:55:12.821535 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.821509    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:55:12.822319 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.822299    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ffcf9bf78-k4pfk"]
Oct 30 21:55:12.837012 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.836985    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837132 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837019    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837132 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837044    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837213 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837134    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837213 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837163    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837213 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837190    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837301 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837252    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.837301 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.837275    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfkn5\" (UniqueName: \"kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.937690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937655    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.937690 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937693    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.937892 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937718    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.937892 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937755    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.937892 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937777    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.937892 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937803    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938088 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937883    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938088 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.937927    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfkn5\" (UniqueName: \"kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938387 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.938333    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938573 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.938547    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938645 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.938589    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938727 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.938707    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.938839 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.938821    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.940493 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.940468    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:12.940675 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:12.940646    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:13.000912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:13.000853    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfkn5\" (UniqueName: \"kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5\") pod \"console-7ffcf9bf78-k4pfk\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") " pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:13.030717 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:13.030687    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:13.409039 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:13.408850    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ffcf9bf78-k4pfk"]
Oct 30 21:55:13.411589 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:55:13.411558    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60443e40_cebd_41c6_b536_6830d50756ea.slice/crio-a9c98dda40ad600ecf012476b6dd75497a1106a6b426b72b12090871ab485035 WatchSource:0}: Error finding container a9c98dda40ad600ecf012476b6dd75497a1106a6b426b72b12090871ab485035: Status 404 returned error can't find the container with id a9c98dda40ad600ecf012476b6dd75497a1106a6b426b72b12090871ab485035
Oct 30 21:55:14.417912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:14.417874    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffcf9bf78-k4pfk" event={"ID":"60443e40-cebd-41c6-b536-6830d50756ea","Type":"ContainerStarted","Data":"a9c98dda40ad600ecf012476b6dd75497a1106a6b426b72b12090871ab485035"}
Oct 30 21:55:16.283328 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.283298    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-rk2x9-n9c9p"]
Oct 30 21:55:16.283877 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.283849    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p" podUID="8b33c42e-93cb-4d53-b533-28c582f874db" containerName="insights-gathering" containerID="cri-o://b99c39a72a008b41b552919d8d4098b625b806be270e249fb2bfaac51def6c8d" gracePeriod=30
Oct 30 21:55:16.426909 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.426881    4536 generic.go:356] "Generic (PLEG): container finished" podID="8b33c42e-93cb-4d53-b533-28c582f874db" containerID="b99c39a72a008b41b552919d8d4098b625b806be270e249fb2bfaac51def6c8d" exitCode=2
Oct 30 21:55:16.427009 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.426933    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p" event={"ID":"8b33c42e-93cb-4d53-b533-28c582f874db","Type":"ContainerDied","Data":"b99c39a72a008b41b552919d8d4098b625b806be270e249fb2bfaac51def6c8d"}
Oct 30 21:55:16.547349 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.547320    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:55:16.670706 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.670672    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") pod \"8b33c42e-93cb-4d53-b533-28c582f874db\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") "
Oct 30 21:55:16.670898 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.670735    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") pod \"8b33c42e-93cb-4d53-b533-28c582f874db\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") "
Oct 30 21:55:16.671076 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.671052    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") pod \"8b33c42e-93cb-4d53-b533-28c582f874db\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") "
Oct 30 21:55:16.671154 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.671097    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle" pod "8b33c42e-93cb-4d53-b533-28c582f874db" (UID: "8b33c42e-93cb-4d53-b533-28c582f874db"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:16.671208 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.671153    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") pod \"8b33c42e-93cb-4d53-b533-28c582f874db\" (UID: \"8b33c42e-93cb-4d53-b533-28c582f874db\") "
Oct 30 21:55:16.671356 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.671342    4536 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-trusted-ca-bundle\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:16.671676 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.671608    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path" pod "8b33c42e-93cb-4d53-b533-28c582f874db" (UID: "8b33c42e-93cb-4d53-b533-28c582f874db"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:16.671792 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.671715    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle" pod "8b33c42e-93cb-4d53-b533-28c582f874db" (UID: "8b33c42e-93cb-4d53-b533-28c582f874db"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:16.673369 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.673341    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm" pod "8b33c42e-93cb-4d53-b533-28c582f874db" (UID: "8b33c42e-93cb-4d53-b533-28c582f874db"). InnerVolumeSpecName "kube-api-access-qrkcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:16.771985 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.771946    4536 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8b33c42e-93cb-4d53-b533-28c582f874db-archives-path\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:16.771985 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.771984    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrkcm\" (UniqueName: \"kubernetes.io/projected/8b33c42e-93cb-4d53-b533-28c582f874db-kube-api-access-qrkcm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:16.772178 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:16.772001    4536 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b33c42e-93cb-4d53-b533-28c582f874db-service-ca-bundle\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:17.431895 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.431800    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffcf9bf78-k4pfk" event={"ID":"60443e40-cebd-41c6-b536-6830d50756ea","Type":"ContainerStarted","Data":"b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a"}
Oct 30 21:55:17.437174 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.437145    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p" event={"ID":"8b33c42e-93cb-4d53-b533-28c582f874db","Type":"ContainerDied","Data":"b679ec01587cf78e7686f0b0b4e0fd1fea94332ec33a13834f4a6ca6d41995a5"}
Oct 30 21:55:17.437300 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.437176    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rk2x9-n9c9p"
Oct 30 21:55:17.437300 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.437223    4536 scope.go:122] "RemoveContainer" containerID="b99c39a72a008b41b552919d8d4098b625b806be270e249fb2bfaac51def6c8d"
Oct 30 21:55:17.552346 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.552285    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7ffcf9bf78-k4pfk" podStartSLOduration=2.541848634 podStartE2EDuration="5.552267797s" totalImagesPullingTime="3.010419163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:17.551685762 +0000 UTC m=+48.539494026" watchObservedRunningTime="2026-10-30 21:55:17.552267797 +0000 UTC m=+48.540076081"
Oct 30 21:55:17.663757 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.663714    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-rk2x9-n9c9p"]
Oct 30 21:55:17.671777 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:17.671747    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-rk2x9-n9c9p"]
Oct 30 21:55:19.162733 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:19.162699    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b33c42e-93cb-4d53-b533-28c582f874db" path="/var/lib/kubelet/pods/8b33c42e-93cb-4d53-b533-28c582f874db/volumes"
Oct 30 21:55:20.803293 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:20.803259    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:55:23.031369 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:23.031336    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:23.031745 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:23.031443    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:23.036151 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:23.036130    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:23.462363 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:23.462335    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:55:28.997863 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:28.997830    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:28.998352 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:28.997893    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:49.003158 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:49.003128    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:49.006918 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:49.006896    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d94746fd5-q662f"
Oct 30 21:55:50.804671 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:50.804635    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:55:57.628539 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.628501    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:57.629568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.628844    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-metric" containerID="cri-o://b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c" gracePeriod=120
Oct 30 21:55:57.629568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.628871    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="prom-label-proxy" containerID="cri-o://932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519" gracePeriod=120
Oct 30 21:55:57.629568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.628913    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="alertmanager" containerID="cri-o://6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce" gracePeriod=120
Oct 30 21:55:57.629568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.629082    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy" containerID="cri-o://3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562" gracePeriod=120
Oct 30 21:55:57.629568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.629090    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-web" containerID="cri-o://46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853" gracePeriod=120
Oct 30 21:55:57.629568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:57.628844    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="config-reloader" containerID="cri-o://a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d" gracePeriod=120
Oct 30 21:55:58.578257 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578225    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519" exitCode=0
Oct 30 21:55:58.578257 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578250    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562" exitCode=0
Oct 30 21:55:58.578257 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578258    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d" exitCode=0
Oct 30 21:55:58.578257 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578264    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce" exitCode=0
Oct 30 21:55:58.578550 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578294    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"}
Oct 30 21:55:58.578550 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578383    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"}
Oct 30 21:55:58.578550 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578432    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"}
Oct 30 21:55:58.578550 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.578457    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"}
Oct 30 21:55:58.875451 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.875427    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:58.980749 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.980719    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.980921 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.980763    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.980921 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.980803    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981047 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.980915    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981047 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.980960    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981047 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981013    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981195 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981057    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981195 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981104    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981195 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981163    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981195 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981192    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981375 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981224    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981375 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981248    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.981375 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981273    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out\") pod \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\" (UID: \"4ef84e48-84e4-4876-8cac-e4f8eed1af3e\") "
Oct 30 21:55:58.982067 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981847    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.982067 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.981853    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:58.982444 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.982422    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-metrics-client-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.982576 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.982561    4536 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-main-db\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.982678 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.982578    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:58.984081 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.984030    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.984248 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.984210    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.984653 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.984626    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "kube-api-access-djqkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:58.984749 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.984688    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.985092 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.985054    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.985354 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.985333    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:58.985579 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.985547    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.985669 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.985645    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.988866 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:58.988845    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:59.013889 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.013839    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config" pod "4ef84e48-84e4-4876-8cac-e4f8eed1af3e" (UID: "4ef84e48-84e4-4876-8cac-e4f8eed1af3e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:59.084020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.083973    4536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084013    4536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-main-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084038    4536 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-web-config\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084048    4536 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-volume\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084058    4536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084068    4536 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-tls-assets\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084076    4536 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084085    4536 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-cluster-tls-config\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084093    4536 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084101    4536 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-config-out\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.084215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.084109    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djqkn\" (UniqueName: \"kubernetes.io/projected/4ef84e48-84e4-4876-8cac-e4f8eed1af3e-kube-api-access-djqkn\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.584985 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.584950    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c" exitCode=0
Oct 30 21:55:59.584985 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.584977    4536 generic.go:356] "Generic (PLEG): container finished" podID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerID="46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853" exitCode=0
Oct 30 21:55:59.585247 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.585024    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"}
Oct 30 21:55:59.585247 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.585046    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.585247 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.585088    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"}
Oct 30 21:55:59.585247 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.585117    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ef84e48-84e4-4876-8cac-e4f8eed1af3e","Type":"ContainerDied","Data":"73e6852597a78a4748c26b54ade0082745f996dc7d6e155fa35e2ba2300655dc"}
Oct 30 21:55:59.585247 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.585148    4536 scope.go:122] "RemoveContainer" containerID="932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"
Oct 30 21:55:59.592432 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.592414    4536 scope.go:122] "RemoveContainer" containerID="b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"
Oct 30 21:55:59.598818 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.598802    4536 scope.go:122] "RemoveContainer" containerID="3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"
Oct 30 21:55:59.605076 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.605059    4536 scope.go:122] "RemoveContainer" containerID="46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"
Oct 30 21:55:59.611487 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.611468    4536 scope.go:122] "RemoveContainer" containerID="a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"
Oct 30 21:55:59.617663 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.617648    4536 scope.go:122] "RemoveContainer" containerID="6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"
Oct 30 21:55:59.624084 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.624064    4536 scope.go:122] "RemoveContainer" containerID="2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"
Oct 30 21:55:59.630210 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630195    4536 scope.go:122] "RemoveContainer" containerID="932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"
Oct 30 21:55:59.630481 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.630462    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519\": container with ID starting with 932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519 not found: ID does not exist" containerID="932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"
Oct 30 21:55:59.630539 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630491    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"} err="failed to get container status \"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519\": rpc error: code = NotFound desc = could not find container \"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519\": container with ID starting with 932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519 not found: ID does not exist"
Oct 30 21:55:59.630539 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630524    4536 scope.go:122] "RemoveContainer" containerID="b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"
Oct 30 21:55:59.630754 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.630740    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c\": container with ID starting with b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c not found: ID does not exist" containerID="b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"
Oct 30 21:55:59.630797 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630758    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"} err="failed to get container status \"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c\": rpc error: code = NotFound desc = could not find container \"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c\": container with ID starting with b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c not found: ID does not exist"
Oct 30 21:55:59.630797 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630770    4536 scope.go:122] "RemoveContainer" containerID="3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"
Oct 30 21:55:59.630979 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.630964    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562\": container with ID starting with 3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562 not found: ID does not exist" containerID="3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"
Oct 30 21:55:59.631019 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630983    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"} err="failed to get container status \"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562\": rpc error: code = NotFound desc = could not find container \"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562\": container with ID starting with 3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562 not found: ID does not exist"
Oct 30 21:55:59.631019 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.630997    4536 scope.go:122] "RemoveContainer" containerID="46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"
Oct 30 21:55:59.631199 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.631183    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853\": container with ID starting with 46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853 not found: ID does not exist" containerID="46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"
Oct 30 21:55:59.631236 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631203    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"} err="failed to get container status \"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853\": rpc error: code = NotFound desc = could not find container \"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853\": container with ID starting with 46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853 not found: ID does not exist"
Oct 30 21:55:59.631236 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631220    4536 scope.go:122] "RemoveContainer" containerID="a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"
Oct 30 21:55:59.631493 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.631474    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d\": container with ID starting with a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d not found: ID does not exist" containerID="a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"
Oct 30 21:55:59.631552 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631498    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"} err="failed to get container status \"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d\": rpc error: code = NotFound desc = could not find container \"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d\": container with ID starting with a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d not found: ID does not exist"
Oct 30 21:55:59.631552 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631513    4536 scope.go:122] "RemoveContainer" containerID="6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"
Oct 30 21:55:59.631756 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.631737    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce\": container with ID starting with 6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce not found: ID does not exist" containerID="6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"
Oct 30 21:55:59.631796 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631761    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"} err="failed to get container status \"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce\": rpc error: code = NotFound desc = could not find container \"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce\": container with ID starting with 6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce not found: ID does not exist"
Oct 30 21:55:59.631796 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631774    4536 scope.go:122] "RemoveContainer" containerID="2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"
Oct 30 21:55:59.631998 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:55:59.631978    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a\": container with ID starting with 2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a not found: ID does not exist" containerID="2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"
Oct 30 21:55:59.632042 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.631997    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"} err="failed to get container status \"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a\": rpc error: code = NotFound desc = could not find container \"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a\": container with ID starting with 2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a not found: ID does not exist"
Oct 30 21:55:59.632042 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632009    4536 scope.go:122] "RemoveContainer" containerID="932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"
Oct 30 21:55:59.632220 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632203    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519"} err="failed to get container status \"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519\": rpc error: code = NotFound desc = could not find container \"932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519\": container with ID starting with 932373c6df6b94160c8a41ff6f1938d58a4ef1b92c7280c611fa17c0843c6519 not found: ID does not exist"
Oct 30 21:55:59.632220 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632218    4536 scope.go:122] "RemoveContainer" containerID="b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"
Oct 30 21:55:59.632458 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632443    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c"} err="failed to get container status \"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c\": rpc error: code = NotFound desc = could not find container \"b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c\": container with ID starting with b44060a1f104d662e9c97884df7569d090521bd153654502bd27d964fa50d52c not found: ID does not exist"
Oct 30 21:55:59.632458 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632457    4536 scope.go:122] "RemoveContainer" containerID="3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"
Oct 30 21:55:59.632685 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632669    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562"} err="failed to get container status \"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562\": rpc error: code = NotFound desc = could not find container \"3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562\": container with ID starting with 3cf61891b6dd94e52dbbc9b9cd2e97b9036d4e3de13d8a32a497825e155d4562 not found: ID does not exist"
Oct 30 21:55:59.632685 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632684    4536 scope.go:122] "RemoveContainer" containerID="46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"
Oct 30 21:55:59.632888 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632874    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853"} err="failed to get container status \"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853\": rpc error: code = NotFound desc = could not find container \"46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853\": container with ID starting with 46cb781f1d7c2b32849a05b4e39d12f7678c21d3459f13f23cf39017cfd59853 not found: ID does not exist"
Oct 30 21:55:59.632888 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.632887    4536 scope.go:122] "RemoveContainer" containerID="a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"
Oct 30 21:55:59.633121 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.633099    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d"} err="failed to get container status \"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d\": rpc error: code = NotFound desc = could not find container \"a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d\": container with ID starting with a95274a83b7d2d7b31b02e13e8e683e42915a96f1c634b40fd2e13b3b349294d not found: ID does not exist"
Oct 30 21:55:59.633173 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.633123    4536 scope.go:122] "RemoveContainer" containerID="6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"
Oct 30 21:55:59.633335 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.633320    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce"} err="failed to get container status \"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce\": rpc error: code = NotFound desc = could not find container \"6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce\": container with ID starting with 6ff58b5a49d330bff853a36910f276b5c952fcb19176ba8e45e09cbfeb267dce not found: ID does not exist"
Oct 30 21:55:59.633372 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.633334    4536 scope.go:122] "RemoveContainer" containerID="2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"
Oct 30 21:55:59.633545 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.633527    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a"} err="failed to get container status \"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a\": rpc error: code = NotFound desc = could not find container \"2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a\": container with ID starting with 2033fcfd544bc8aa66e5a6d2bcd0f25d1a8e210d31085dd8aac5adfd49f4ae3a not found: ID does not exist"
Oct 30 21:55:59.753316 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.753293    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:59.803460 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.803428    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:59.804215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804191    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:59.804481 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804465    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804483    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804502    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="prom-label-proxy"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804510    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="prom-label-proxy"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804524    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="alertmanager"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804531    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="alertmanager"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804542    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-web"
Oct 30 21:55:59.804558 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804549    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-web"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804609    4536 cpu_manager.go:414] "containerMap: removing container" podUID="8b33c42e-93cb-4d53-b533-28c582f874db" containerName="insights-gathering"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804620    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b33c42e-93cb-4d53-b533-28c582f874db" containerName="insights-gathering"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804634    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="init-config-reloader"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804643    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="init-config-reloader"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804654    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="config-reloader"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804661    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="config-reloader"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804670    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-metric"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804678    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-metric"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804746    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="config-reloader"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804757    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-metric"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804768    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804785    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="prom-label-proxy"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804794    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="alertmanager"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804802    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" containerName="kube-rbac-proxy-web"
Oct 30 21:55:59.804912 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.804814    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b33c42e-93cb-4d53-b533-28c582f874db" containerName="insights-gathering"
Oct 30 21:55:59.808057 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.808038    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.861932 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.861872    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:55:59.862159 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862127    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:55:59.862286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862184    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:55:59.862286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862193    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:55:59.862286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862196    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:55:59.862286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862242    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:55:59.862286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862253    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:55:59.862286 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862244    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:55:59.862648 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.862314    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5n786\""
Oct 30 21:55:59.889392 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889366    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889764 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889424    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889764 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889527    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-config-volume\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889764 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889563    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxdv8\" (UniqueName: \"kubernetes.io/projected/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-kube-api-access-sxdv8\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889764 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889622    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889764 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889648    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889764 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889669    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889768    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-web-config\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889801    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889836    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889853    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889894    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-config-out\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.889998 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.889951    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.911111 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.911089    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:55:59.991236 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991206    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991426 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991239    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991426 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991258    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991426 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991284    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-web-config\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991426 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991338    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991426 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991379    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991426 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991425    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991750 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991454    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-config-out\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991750 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991496    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991750 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991522    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991750 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991539    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991750 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991560    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-config-volume\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.991750 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991593    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxdv8\" (UniqueName: \"kubernetes.io/projected/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-kube-api-access-sxdv8\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.992062 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.991879    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.992624 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.992532    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.993196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.993169    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.994641 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.994596    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.994754 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.994734    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.994968 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.994948    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-config-volume\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.995046 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.994950    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-web-config\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.995308 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.995284    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.995436 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.995334    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.995547 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.995525    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.995834 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.995808    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-config-out\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:59.996722 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:55:59.996703    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:56:00.044729 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:00.044704    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:56:00.053942 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:00.053921    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxdv8\" (UniqueName: \"kubernetes.io/projected/1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9-kube-api-access-sxdv8\") pod \"alertmanager-main-0\" (UID: \"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:56:00.123326 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:00.123262    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:56:00.556058 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:00.555940    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:56:00.558490 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:56:00.558467    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3f2e36_f4a8_4f87_8c35_c8da2c5cd3a9.slice/crio-7330f91e5c0827aa9d8893d43b1432d17fff297188d8faa3f4c96b33c2cc6191 WatchSource:0}: Error finding container 7330f91e5c0827aa9d8893d43b1432d17fff297188d8faa3f4c96b33c2cc6191: Status 404 returned error can't find the container with id 7330f91e5c0827aa9d8893d43b1432d17fff297188d8faa3f4c96b33c2cc6191
Oct 30 21:56:00.590896 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:00.590866    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"7330f91e5c0827aa9d8893d43b1432d17fff297188d8faa3f4c96b33c2cc6191"}
Oct 30 21:56:01.162659 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.162628    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ef84e48-84e4-4876-8cac-e4f8eed1af3e" path="/var/lib/kubelet/pods/4ef84e48-84e4-4876-8cac-e4f8eed1af3e/volumes"
Oct 30 21:56:01.293867 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.293828    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76bc55d459-9w2j9"]
Oct 30 21:56:01.296215 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.296192    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.350813 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.350774    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:56:01.350813 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.350789    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:56:01.350813 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.350813    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:56:01.351093 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.350930    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:56:01.400138 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400111    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8t826\""
Oct 30 21:56:01.400518 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400415    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400545    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-metrics-client-ca\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400606    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-telemeter-client-tls\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400691    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-secret-telemeter-client\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400720    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-federate-client-tls\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400746    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400827    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddjgq\" (UniqueName: \"kubernetes.io/projected/04634017-52b0-44dc-9cb2-a75785a88081-kube-api-access-ddjgq\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.400875 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400849    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-serving-certs-ca-bundle\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.401240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.400943    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.450704 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.450670    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:56:01.502177 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502146    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddjgq\" (UniqueName: \"kubernetes.io/projected/04634017-52b0-44dc-9cb2-a75785a88081-kube-api-access-ddjgq\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502182    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-serving-certs-ca-bundle\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502208    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502233    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-metrics-client-ca\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502259    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-telemeter-client-tls\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502276    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-secret-telemeter-client\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502291    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-federate-client-tls\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.502323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502309    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.503019 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.502987    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-serving-certs-ca-bundle\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.503140 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.503040    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-metrics-client-ca\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.503468 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.503441    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04634017-52b0-44dc-9cb2-a75785a88081-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.505026 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.504998    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-federate-client-tls\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.505115 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.505081    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-secret-telemeter-client\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.505330 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.505308    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.505500 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.505484    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/04634017-52b0-44dc-9cb2-a75785a88081-telemeter-client-tls\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.518623 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.518599    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76bc55d459-9w2j9"]
Oct 30 21:56:01.564820 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.564790    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddjgq\" (UniqueName: \"kubernetes.io/projected/04634017-52b0-44dc-9cb2-a75785a88081-kube-api-access-ddjgq\") pod \"telemeter-client-76bc55d459-9w2j9\" (UID: \"04634017-52b0-44dc-9cb2-a75785a88081\") " pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:01.596580 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.596547    4536 generic.go:356] "Generic (PLEG): container finished" podID="1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9" containerID="97cc19e73de4dbb1b2be5a5aee428ad588a0691c405ee956f9ed3e1db2978bdc" exitCode=0
Oct 30 21:56:01.596736 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.596621    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerDied","Data":"97cc19e73de4dbb1b2be5a5aee428ad588a0691c405ee956f9ed3e1db2978bdc"}
Oct 30 21:56:01.608568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:01.608549    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9"
Oct 30 21:56:02.035831 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.035794    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76bc55d459-9w2j9"]
Oct 30 21:56:02.037933 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:56:02.037901    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04634017_52b0_44dc_9cb2_a75785a88081.slice/crio-47dc5fb44daa4b75d6954355ff0297b428db41dbc894dde468529421b03ee771 WatchSource:0}: Error finding container 47dc5fb44daa4b75d6954355ff0297b428db41dbc894dde468529421b03ee771: Status 404 returned error can't find the container with id 47dc5fb44daa4b75d6954355ff0297b428db41dbc894dde468529421b03ee771
Oct 30 21:56:02.603545 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.603505    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"0c5b2bc45b5f60c949e3f3430da8a446d25934ba0289fec37f9073cc6384d78f"}
Oct 30 21:56:02.604043 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.603610    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"869583514eecc7bfb0d2999cd0edaf889c7208ceacc2389589c9f2c5b39f0cc6"}
Oct 30 21:56:02.604043 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.603641    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"c20435a59845e6e561bc72675b73cf139fe91628abbe4e8445f7e86f5597c85d"}
Oct 30 21:56:02.604043 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.603668    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"014eeceac8c9710fbbbcc07308f651e9bffcbce6bd45b50bb6eec4da95b80ea8"}
Oct 30 21:56:02.604043 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.603702    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"98dd00f97b8fa0f66ea02f19b4d08934296cea5f0c08b8e1119a731a0b67535c"}
Oct 30 21:56:02.604043 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.603725    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9","Type":"ContainerStarted","Data":"893b2c89157f4bc3f06caaf1dbbfea62707ef6a9bfd955955a1b0cdae433cafc"}
Oct 30 21:56:02.604725 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.604703    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9" event={"ID":"04634017-52b0-44dc-9cb2-a75785a88081","Type":"ContainerStarted","Data":"47dc5fb44daa4b75d6954355ff0297b428db41dbc894dde468529421b03ee771"}
Oct 30 21:56:02.738387 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:02.738339    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.724888335 podStartE2EDuration="3.738321446s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.433111ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:02.737331816 +0000 UTC m=+93.725140090" watchObservedRunningTime="2026-10-30 21:56:02.738321446 +0000 UTC m=+93.726129710"
Oct 30 21:56:04.614432 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:04.614382    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9" event={"ID":"04634017-52b0-44dc-9cb2-a75785a88081","Type":"ContainerStarted","Data":"7ce2f903b85138e18fb73f50f35d64b2c3ea0b5c7afc560a5b8b8b9603bba73b"}
Oct 30 21:56:04.614831 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:04.614449    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9" event={"ID":"04634017-52b0-44dc-9cb2-a75785a88081","Type":"ContainerStarted","Data":"796dc39484ad04afcefe67d967ce1f1d159a602d77b03f85b13a111c4f1cc4b7"}
Oct 30 21:56:04.614831 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:04.614468    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9" event={"ID":"04634017-52b0-44dc-9cb2-a75785a88081","Type":"ContainerStarted","Data":"0e7040f537080a9e42eb1354b84e98b76eb1bdbad6c13c9bd9bd78ce53382ca2"}
Oct 30 21:56:04.839085 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:04.839031    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76bc55d459-9w2j9" podStartSLOduration=1.831001509 podStartE2EDuration="3.83901644s" totalImagesPullingTime="2.008014931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:04.788858197 +0000 UTC m=+95.776666461" watchObservedRunningTime="2026-10-30 21:56:04.83901644 +0000 UTC m=+95.826824703"
Oct 30 21:56:05.228563 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.228526    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-65b579bc9d-mps8m"]
Oct 30 21:56:05.231824 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.231801    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.294068 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.294039    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b579bc9d-mps8m"]
Oct 30 21:56:05.334910 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.334881    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335057 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.334918    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxxv4\" (UniqueName: \"kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335057 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.334939    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335057 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.334955    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335203 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.335084    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335203 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.335109    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335203 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.335134    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.335307 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.335284    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436149 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436120    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436149 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436155    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436347 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436185    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxxv4\" (UniqueName: \"kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436347 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436212    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436347 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436324    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436528 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436430    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436528 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436466    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436528 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436499    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.436929 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.436897    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.437049 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.437010    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.437049 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.437027    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.437160 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.437053    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.437430 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.437385    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.443259 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.439208    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.443259 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.439390    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.499105 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.499041    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxxv4\" (UniqueName: \"kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4\") pod \"console-65b579bc9d-mps8m\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") " pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.543187 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.543148    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:05.919208 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:05.919139    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b579bc9d-mps8m"]
Oct 30 21:56:05.944868 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:56:05.944840    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69e2ff3_69ee_4464_ace2_327394c27cd4.slice/crio-5fef8d72c22a2ce80ad95da4eb6e88f06ad12a8d980dfcfc40eeaa2fe02f3754 WatchSource:0}: Error finding container 5fef8d72c22a2ce80ad95da4eb6e88f06ad12a8d980dfcfc40eeaa2fe02f3754: Status 404 returned error can't find the container with id 5fef8d72c22a2ce80ad95da4eb6e88f06ad12a8d980dfcfc40eeaa2fe02f3754
Oct 30 21:56:06.624660 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:06.624624    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b579bc9d-mps8m" event={"ID":"c69e2ff3-69ee-4464-ace2-327394c27cd4","Type":"ContainerStarted","Data":"21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d"}
Oct 30 21:56:06.624833 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:06.624675    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b579bc9d-mps8m" event={"ID":"c69e2ff3-69ee-4464-ace2-327394c27cd4","Type":"ContainerStarted","Data":"5fef8d72c22a2ce80ad95da4eb6e88f06ad12a8d980dfcfc40eeaa2fe02f3754"}
Oct 30 21:56:06.756482 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:06.756429    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-65b579bc9d-mps8m" podStartSLOduration=1.756390924 podStartE2EDuration="1.756390924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:06.755252734 +0000 UTC m=+97.743060997" watchObservedRunningTime="2026-10-30 21:56:06.756390924 +0000 UTC m=+97.744199186"
Oct 30 21:56:15.544245 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:15.544200    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:15.544686 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:15.544269    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:15.549611 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:15.549589    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:15.658487 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:15.658459    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:56:15.831320 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:15.831252    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ffcf9bf78-k4pfk"]
Oct 30 21:56:20.804100 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:56:20.804060    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:56:40.851041 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:40.850938    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7ffcf9bf78-k4pfk" podUID="60443e40-cebd-41c6-b536-6830d50756ea" containerName="console" containerID="cri-o://b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a" gracePeriod=15
Oct 30 21:56:41.097939 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.097914    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:56:41.124313 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124267    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124313 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124298    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124486 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124317    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124486 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124336    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124486 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124379    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124486 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124441    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124486 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124482    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp\" (UniqueName: \"kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124742 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124515    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5\" (UniqueName: \"kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5\") pod \"60443e40-cebd-41c6-b536-6830d50756ea\" (UID: \"60443e40-cebd-41c6-b536-6830d50756ea\") "
Oct 30 21:56:41.124806 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124778    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:41.124806 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124789    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:41.124919 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124808    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:41.124919 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.124863    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:41.125081 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.125062    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:41.126705 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.126676    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:41.126840 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.126714    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:41.126840 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.126735    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5" pod "60443e40-cebd-41c6-b536-6830d50756ea" (UID: "60443e40-cebd-41c6-b536-6830d50756ea"). InnerVolumeSpecName "kube-api-access-xfkn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:41.225056 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225024    4536 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-serving-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225056 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225054    4536 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-oauth-serving-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225067    4536 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60443e40-cebd-41c6-b536-6830d50756ea-tmp\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225079    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfkn5\" (UniqueName: \"kubernetes.io/projected/60443e40-cebd-41c6-b536-6830d50756ea-kube-api-access-xfkn5\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225091    4536 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60443e40-cebd-41c6-b536-6830d50756ea-console-oauth-config\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225105    4536 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-trusted-ca-bundle\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225119    4536 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.225240 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.225133    4536 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60443e40-cebd-41c6-b536-6830d50756ea-console-config\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.740563 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.740526    4536 generic.go:356] "Generic (PLEG): container finished" podID="60443e40-cebd-41c6-b536-6830d50756ea" containerID="b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a" exitCode=0
Oct 30 21:56:41.740745 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.740577    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffcf9bf78-k4pfk"
Oct 30 21:56:41.740745 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.740610    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffcf9bf78-k4pfk" event={"ID":"60443e40-cebd-41c6-b536-6830d50756ea","Type":"ContainerDied","Data":"b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a"}
Oct 30 21:56:41.740745 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.740661    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffcf9bf78-k4pfk" event={"ID":"60443e40-cebd-41c6-b536-6830d50756ea","Type":"ContainerDied","Data":"a9c98dda40ad600ecf012476b6dd75497a1106a6b426b72b12090871ab485035"}
Oct 30 21:56:41.740745 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.740684    4536 scope.go:122] "RemoveContainer" containerID="b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a"
Oct 30 21:56:41.748449 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.748420    4536 scope.go:122] "RemoveContainer" containerID="b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a"
Oct 30 21:56:41.748695 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:56:41.748676    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a\": container with ID starting with b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a not found: ID does not exist" containerID="b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a"
Oct 30 21:56:41.748749 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.748705    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a"} err="failed to get container status \"b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a\": rpc error: code = NotFound desc = could not find container \"b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a\": container with ID starting with b64cb0af32bd31c0740fba89745e7b5c44f5ae901fbf02588f46d25d45dda91a not found: ID does not exist"
Oct 30 21:56:41.907414 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.907359    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ffcf9bf78-k4pfk"]
Oct 30 21:56:41.956977 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:41.956943    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ffcf9bf78-k4pfk"]
Oct 30 21:56:43.161779 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:56:43.161746    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60443e40-cebd-41c6-b536-6830d50756ea" path="/var/lib/kubelet/pods/60443e40-cebd-41c6-b536-6830d50756ea/volumes"
Oct 30 21:56:50.803328 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:56:50.803290    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:57:14.088947 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.088916    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c69cfc66-9d7js"]
Oct 30 21:57:14.089449 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.089199    4536 cpu_manager.go:414] "containerMap: removing container" podUID="60443e40-cebd-41c6-b536-6830d50756ea" containerName="console"
Oct 30 21:57:14.089449 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.089211    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60443e40-cebd-41c6-b536-6830d50756ea" containerName="console"
Oct 30 21:57:14.089449 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.089289    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="60443e40-cebd-41c6-b536-6830d50756ea" containerName="console"
Oct 30 21:57:14.092130 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.092115    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153077 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153045    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c69cfc66-9d7js"]
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153163    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-trusted-ca-bundle\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153204    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-service-ca\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153231    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-oauth-config\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153325    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-oauth-serving-cert\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153354    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-config\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153372    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb61d1d2-e276-4d43-9c3b-d76633b5f695-tmp\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153419 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153419    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl76s\" (UniqueName: \"kubernetes.io/projected/eb61d1d2-e276-4d43-9c3b-d76633b5f695-kube-api-access-cl76s\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.153734 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.153539    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-serving-cert\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.253930 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.253893    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-serving-cert\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254117 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.253950    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-trusted-ca-bundle\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254117 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254070    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-service-ca\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254117 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254115    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-oauth-config\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254287 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254151    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-oauth-serving-cert\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254287 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254187    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-config\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254287 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254209    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb61d1d2-e276-4d43-9c3b-d76633b5f695-tmp\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254287 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254237    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl76s\" (UniqueName: \"kubernetes.io/projected/eb61d1d2-e276-4d43-9c3b-d76633b5f695-kube-api-access-cl76s\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254756 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254732    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb61d1d2-e276-4d43-9c3b-d76633b5f695-tmp\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254856 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254834    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-trusted-ca-bundle\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254914 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254896    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-service-ca\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.254967 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.254943    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-config\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.255196 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.255180    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb61d1d2-e276-4d43-9c3b-d76633b5f695-oauth-serving-cert\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.257206 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.257180    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-serving-cert\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.257323 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.257299    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb61d1d2-e276-4d43-9c3b-d76633b5f695-console-oauth-config\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.317258 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.317232    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl76s\" (UniqueName: \"kubernetes.io/projected/eb61d1d2-e276-4d43-9c3b-d76633b5f695-kube-api-access-cl76s\") pod \"console-5c69cfc66-9d7js\" (UID: \"eb61d1d2-e276-4d43-9c3b-d76633b5f695\") " pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.403375 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.403299    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:14.776691 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.776657    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c69cfc66-9d7js"]
Oct 30 21:57:14.808233 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:57:14.808204    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb61d1d2_e276_4d43_9c3b_d76633b5f695.slice/crio-7c6e8fc059bfefb256ff0421aaa669e0c5c413d3472eb4cf7364fc7e9dc604d8 WatchSource:0}: Error finding container 7c6e8fc059bfefb256ff0421aaa669e0c5c413d3472eb4cf7364fc7e9dc604d8: Status 404 returned error can't find the container with id 7c6e8fc059bfefb256ff0421aaa669e0c5c413d3472eb4cf7364fc7e9dc604d8
Oct 30 21:57:14.858377 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:14.858341    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c69cfc66-9d7js" event={"ID":"eb61d1d2-e276-4d43-9c3b-d76633b5f695","Type":"ContainerStarted","Data":"7c6e8fc059bfefb256ff0421aaa669e0c5c413d3472eb4cf7364fc7e9dc604d8"}
Oct 30 21:57:15.863087 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:15.863054    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c69cfc66-9d7js" event={"ID":"eb61d1d2-e276-4d43-9c3b-d76633b5f695","Type":"ContainerStarted","Data":"0b1aaae8ae6e9c0109dcbc5ee6dfa0ffedd8ffbb4f158a6c5535951629d49503"}
Oct 30 21:57:15.984683 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:15.984631    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5c69cfc66-9d7js" podStartSLOduration=1.984617651 podStartE2EDuration="1.984617651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:15.983203872 +0000 UTC m=+166.971012136" watchObservedRunningTime="2026-10-30 21:57:15.984617651 +0000 UTC m=+166.972425914"
Oct 30 21:57:20.803940 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:57:20.803903    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:57:24.403477 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:24.403435    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:24.403929 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:24.403508    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:24.408383 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:24.408362    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:24.897243 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:24.897214    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c69cfc66-9d7js"
Oct 30 21:57:25.068312 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:25.068277    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b579bc9d-mps8m"]
Oct 30 21:57:50.088257 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.088197    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-65b579bc9d-mps8m" podUID="c69e2ff3-69ee-4464-ace2-327394c27cd4" containerName="console" containerID="cri-o://21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d" gracePeriod=15
Oct 30 21:57:50.328221 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.328193    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:57:50.446898 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.446867    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.446917    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4\" (UniqueName: \"kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.446945    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.446976    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447017    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447034    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447074 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447055    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447381 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447091    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp\") pod \"c69e2ff3-69ee-4464-ace2-327394c27cd4\" (UID: \"c69e2ff3-69ee-4464-ace2-327394c27cd4\") "
Oct 30 21:57:50.447503 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447476    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:50.447561 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447489    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:50.447561 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447471    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:50.447561 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447522    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:50.447969 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.447946    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:50.449423 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.449381    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "kube-api-access-sxxv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:50.449521 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.449498    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:50.449948 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.449916    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config" pod "c69e2ff3-69ee-4464-ace2-327394c27cd4" (UID: "c69e2ff3-69ee-4464-ace2-327394c27cd4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:50.548153 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548110    4536 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-oauth-serving-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548153 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548149    4536 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-config\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548153 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548158    4536 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-oauth-config\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548422 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548167    4536 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c69e2ff3-69ee-4464-ace2-327394c27cd4-tmp\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548422 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548176    4536 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c69e2ff3-69ee-4464-ace2-327394c27cd4-console-serving-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548422 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548184    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxxv4\" (UniqueName: \"kubernetes.io/projected/c69e2ff3-69ee-4464-ace2-327394c27cd4-kube-api-access-sxxv4\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548422 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548194    4536 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-trusted-ca-bundle\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.548422 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.548202    4536 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c69e2ff3-69ee-4464-ace2-327394c27cd4-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.805309 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:57:50.805221    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:57:50.981784 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.981756    4536 generic.go:356] "Generic (PLEG): container finished" podID="c69e2ff3-69ee-4464-ace2-327394c27cd4" containerID="21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d" exitCode=0
Oct 30 21:57:50.981953 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.981803    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b579bc9d-mps8m" event={"ID":"c69e2ff3-69ee-4464-ace2-327394c27cd4","Type":"ContainerDied","Data":"21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d"}
Oct 30 21:57:50.981953 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.981839    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b579bc9d-mps8m" event={"ID":"c69e2ff3-69ee-4464-ace2-327394c27cd4","Type":"ContainerDied","Data":"5fef8d72c22a2ce80ad95da4eb6e88f06ad12a8d980dfcfc40eeaa2fe02f3754"}
Oct 30 21:57:50.981953 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.981859    4536 scope.go:122] "RemoveContainer" containerID="21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d"
Oct 30 21:57:50.981953 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.981806    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b579bc9d-mps8m"
Oct 30 21:57:50.990015 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.989998    4536 scope.go:122] "RemoveContainer" containerID="21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d"
Oct 30 21:57:50.990277 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:57:50.990259    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d\": container with ID starting with 21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d not found: ID does not exist" containerID="21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d"
Oct 30 21:57:50.990322 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:50.990287    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d"} err="failed to get container status \"21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d\": rpc error: code = NotFound desc = could not find container \"21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d\": container with ID starting with 21674d40aa959dfb155b1792382c2703800a4bf7db9fb50862b9fe678493310d not found: ID does not exist"
Oct 30 21:57:51.147910 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:51.147825    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b579bc9d-mps8m"]
Oct 30 21:57:51.197618 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:51.197582    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65b579bc9d-mps8m"]
Oct 30 21:57:53.161984 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:57:53.161940    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c69e2ff3-69ee-4464-ace2-327394c27cd4" path="/var/lib/kubelet/pods/c69e2ff3-69ee-4464-ace2-327394c27cd4/volumes"
Oct 30 21:58:20.804822 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:58:20.804747    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:58:50.803864 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:58:50.803821    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:59:20.804451 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:59:20.804415    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:59:50.804223 ip-10-0-138-166 kubenswrapper[4536]: E1030 21:59:50.804138    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 21:59:50.844501 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.844468    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"]
Oct 30 21:59:50.844782 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.844771    4536 cpu_manager.go:414] "containerMap: removing container" podUID="c69e2ff3-69ee-4464-ace2-327394c27cd4" containerName="console"
Oct 30 21:59:50.844823 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.844784    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c69e2ff3-69ee-4464-ace2-327394c27cd4" containerName="console"
Oct 30 21:59:50.844855 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.844831    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="c69e2ff3-69ee-4464-ace2-327394c27cd4" containerName="console"
Oct 30 21:59:50.847815 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.847800    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:50.950613 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.950573    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-qnd8v\""
Oct 30 21:59:50.950613 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.950587    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:59:50.950841 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.950723    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:59:50.951736 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.951711    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"]
Oct 30 21:59:50.954203 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.954183    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:50.954259 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.954221    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:50.954259 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:50.954245    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncd8z\" (UniqueName: \"kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.055703 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.055622    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.055703 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.055663    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncd8z\" (UniqueName: \"kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.055901 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.055736    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.056108 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.056092    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.056172 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.056092    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.101935 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.101898    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"]
Oct 30 21:59:51.105034 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.105019    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.120115 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.120076    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncd8z\" (UniqueName: \"kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.157014 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.156980    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.157161 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.157029    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.157161 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.157126    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjzwk\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.157231 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.157195    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.157231 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.157212    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.157318 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.157229    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.157318 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.157266    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.160221 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160201    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:59:51.160384 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160236    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:59:51.160384 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160236    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-x7p2g\""
Oct 30 21:59:51.160384 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160316    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:59:51.160635 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160340    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:59:51.160635 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160430    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:59:51.160635 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.160468    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:59:51.163502 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.163487    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 21:59:51.169830 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.169797    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"]
Oct 30 21:59:51.257867 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.257839    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.258020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.257878    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.258020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.257923    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjzwk\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.258020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.257969    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.258020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.257994    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.258020 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.258019    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.258267 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.258067    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.259046 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.259016    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.260535 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.260508    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.260639 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.260533    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.260639 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.260572    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.260639 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.260537    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.321471 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.321389    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjzwk\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.323138 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.323120    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-n9mts\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.417525 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.417499    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:51.537812 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.537788    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"]
Oct 30 21:59:51.540475 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:59:51.540445    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf9deff_cdbc_487d_b706_213b8a71fdee.slice/crio-749d032ddef76504be40a39d9b5433bd6d68b323a5b3e5f59f77a477852352b7 WatchSource:0}: Error finding container 749d032ddef76504be40a39d9b5433bd6d68b323a5b3e5f59f77a477852352b7: Status 404 returned error can't find the container with id 749d032ddef76504be40a39d9b5433bd6d68b323a5b3e5f59f77a477852352b7
Oct 30 21:59:51.542303 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.542288    4536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:59:51.793108 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:51.793076    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"]
Oct 30 21:59:51.795753 ip-10-0-138-166 kubenswrapper[4536]: W1030 21:59:51.795721    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2efb62_29a0_4c63_b743_e9ded5ed22bb.slice/crio-d8131fd794eef8029504fd5f895c950a229f90daa0b49ba2cf613c744aac0503 WatchSource:0}: Error finding container d8131fd794eef8029504fd5f895c950a229f90daa0b49ba2cf613c744aac0503: Status 404 returned error can't find the container with id d8131fd794eef8029504fd5f895c950a229f90daa0b49ba2cf613c744aac0503
Oct 30 21:59:52.392755 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:52.392720    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" event={"ID":"6b2efb62-29a0-4c63-b743-e9ded5ed22bb","Type":"ContainerStarted","Data":"d8131fd794eef8029504fd5f895c950a229f90daa0b49ba2cf613c744aac0503"}
Oct 30 21:59:52.393964 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:52.393943    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4" event={"ID":"dbf9deff-cdbc-487d-b706-213b8a71fdee","Type":"ContainerStarted","Data":"749d032ddef76504be40a39d9b5433bd6d68b323a5b3e5f59f77a477852352b7"}
Oct 30 21:59:55.760080 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:55.760042    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 21:59:55.760462 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:55.760109    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 21:59:56.413026 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:56.412990    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" event={"ID":"6b2efb62-29a0-4c63-b743-e9ded5ed22bb","Type":"ContainerStarted","Data":"5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299"}
Oct 30 21:59:56.413366 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:56.413344    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:56.415274 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:56.415244    4536 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-n9mts container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:59:56.415384 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:56.415302    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:59:56.624568 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:56.624503    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" podStartSLOduration=1.662886763 podStartE2EDuration="5.624483722s" totalImagesPullingTime="3.961596959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:56.590471353 +0000 UTC m=+327.578279610" watchObservedRunningTime="2026-10-30 21:59:56.624483722 +0000 UTC m=+327.612291986"
Oct 30 21:59:57.418334 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:57.418306    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 21:59:59.426048 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:59.426009    4536 generic.go:356] "Generic (PLEG): container finished" podID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerID="a2b99a5b135c6028c682e08f646075cb2b9110f8a6879c1921accd16a3afe2e5" exitCode=0
Oct 30 21:59:59.426451 ip-10-0-138-166 kubenswrapper[4536]: I1030 21:59:59.426100    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4" event={"ID":"dbf9deff-cdbc-487d-b706-213b8a71fdee","Type":"ContainerDied","Data":"a2b99a5b135c6028c682e08f646075cb2b9110f8a6879c1921accd16a3afe2e5"}
Oct 30 22:00:03.444903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:03.444866    4536 generic.go:356] "Generic (PLEG): container finished" podID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerID="2bf99cf0928a621c11f3d0a26c8c90da7dd936eb543b52bcd4fdb982a83e6791" exitCode=0
Oct 30 22:00:03.445346 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:03.444949    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4" event={"ID":"dbf9deff-cdbc-487d-b706-213b8a71fdee","Type":"ContainerDied","Data":"2bf99cf0928a621c11f3d0a26c8c90da7dd936eb543b52bcd4fdb982a83e6791"}
Oct 30 22:00:20.807221 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:00:20.807182    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:00:23.519542 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:23.519503    4536 generic.go:356] "Generic (PLEG): container finished" podID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerID="2f9c854869a8ca68cb7d00f565d4eafb594a2738d1ad405ca6fbb98438a1941f" exitCode=0
Oct 30 22:00:23.519955 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:23.519572    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4" event={"ID":"dbf9deff-cdbc-487d-b706-213b8a71fdee","Type":"ContainerDied","Data":"2f9c854869a8ca68cb7d00f565d4eafb594a2738d1ad405ca6fbb98438a1941f"}
Oct 30 22:00:24.644302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.644280    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 22:00:24.745195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.745160    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z\" (UniqueName: \"kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z\") pod \"dbf9deff-cdbc-487d-b706-213b8a71fdee\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") "
Oct 30 22:00:24.745332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.745261    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util\") pod \"dbf9deff-cdbc-487d-b706-213b8a71fdee\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") "
Oct 30 22:00:24.745332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.745288    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle\") pod \"dbf9deff-cdbc-487d-b706-213b8a71fdee\" (UID: \"dbf9deff-cdbc-487d-b706-213b8a71fdee\") "
Oct 30 22:00:24.745738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.745706    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle" pod "dbf9deff-cdbc-487d-b706-213b8a71fdee" (UID: "dbf9deff-cdbc-487d-b706-213b8a71fdee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:24.747576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.747546    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z" pod "dbf9deff-cdbc-487d-b706-213b8a71fdee" (UID: "dbf9deff-cdbc-487d-b706-213b8a71fdee"). InnerVolumeSpecName "kube-api-access-ncd8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:24.750641 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.750614    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util" pod "dbf9deff-cdbc-487d-b706-213b8a71fdee" (UID: "dbf9deff-cdbc-487d-b706-213b8a71fdee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:24.846601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.846509    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncd8z\" (UniqueName: \"kubernetes.io/projected/dbf9deff-cdbc-487d-b706-213b8a71fdee-kube-api-access-ncd8z\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:24.846601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.846551    4536 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-util\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:24.846601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:24.846560    4536 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf9deff-cdbc-487d-b706-213b8a71fdee-bundle\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:25.527730 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:25.527699    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4" event={"ID":"dbf9deff-cdbc-487d-b706-213b8a71fdee","Type":"ContainerDied","Data":"749d032ddef76504be40a39d9b5433bd6d68b323a5b3e5f59f77a477852352b7"}
Oct 30 22:00:25.527895 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:25.527743    4536 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="749d032ddef76504be40a39d9b5433bd6d68b323a5b3e5f59f77a477852352b7"
Oct 30 22:00:25.527895 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:25.527745    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6445vjn4"
Oct 30 22:00:46.076378 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076337    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"]
Oct 30 22:00:46.076811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076726    4536 cpu_manager.go:414] "containerMap: removing container" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="extract"
Oct 30 22:00:46.076811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076739    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="extract"
Oct 30 22:00:46.076811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076751    4536 cpu_manager.go:414] "containerMap: removing container" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="pull"
Oct 30 22:00:46.076811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076755    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="pull"
Oct 30 22:00:46.076811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076775    4536 cpu_manager.go:414] "containerMap: removing container" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="util"
Oct 30 22:00:46.076811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076779    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="util"
Oct 30 22:00:46.077024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.076837    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbf9deff-cdbc-487d-b706-213b8a71fdee" containerName="extract"
Oct 30 22:00:46.078627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.078604    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.123295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.123256    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkr7s\" (UniqueName: \"kubernetes.io/projected/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-kube-api-access-lkr7s\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.123479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.123304    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.123479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.123329    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.123479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.123443    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.123603 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.123484    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-manifests\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.123603 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.123564    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-tmp\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.194871 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.194839    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 22:00:46.195039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.194917    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 22:00:46.195039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.195024    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 22:00:46.195115 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.195043    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-9rb8f\""
Oct 30 22:00:46.199902 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.199887    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 22:00:46.199957 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.199891    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 22:00:46.209372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.209347    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"]
Oct 30 22:00:46.224956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.224927    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.225098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.224962    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.225098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.224987    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-manifests\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.225098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.225076    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-tmp\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.225254 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.225219    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkr7s\" (UniqueName: \"kubernetes.io/projected/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-kube-api-access-lkr7s\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.225308 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.225276    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.225568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.225539    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-manifests\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.227586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.227566    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-tmp\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.227688 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.227673    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.228224 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.228202    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.228296 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.228202    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.288382 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.288355    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkr7s\" (UniqueName: \"kubernetes.io/projected/9fdbd49b-36f2-4aeb-bc70-06f8647d2934-kube-api-access-lkr7s\") pod \"opendatahub-operator-controller-manager-f7fbfb566-jwxmf\" (UID: \"9fdbd49b-36f2-4aeb-bc70-06f8647d2934\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.390758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.390669    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:46.878678 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:46.878654    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"]
Oct 30 22:00:46.880619 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:00:46.880589    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fdbd49b_36f2_4aeb_bc70_06f8647d2934.slice/crio-b7c3e5d4a091b7a41bada2ed62fe531a96f5426c288d871b0fe82a8b38dacea0 WatchSource:0}: Error finding container b7c3e5d4a091b7a41bada2ed62fe531a96f5426c288d871b0fe82a8b38dacea0: Status 404 returned error can't find the container with id b7c3e5d4a091b7a41bada2ed62fe531a96f5426c288d871b0fe82a8b38dacea0
Oct 30 22:00:47.606880 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:47.606840    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf" event={"ID":"9fdbd49b-36f2-4aeb-bc70-06f8647d2934","Type":"ContainerStarted","Data":"b7c3e5d4a091b7a41bada2ed62fe531a96f5426c288d871b0fe82a8b38dacea0"}
Oct 30 22:00:50.621863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:50.621825    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf" event={"ID":"9fdbd49b-36f2-4aeb-bc70-06f8647d2934","Type":"ContainerStarted","Data":"d6b082aa6f2c69faeb7ddb721d31f5f0039b2532b8ffd6bbfe8746651da4d38f"}
Oct 30 22:00:50.805079 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:00:50.805042    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:00:51.626890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:51.626855    4536 generic.go:356] "Generic (PLEG): container finished" podID="9fdbd49b-36f2-4aeb-bc70-06f8647d2934" containerID="d6b082aa6f2c69faeb7ddb721d31f5f0039b2532b8ffd6bbfe8746651da4d38f" exitCode=0
Oct 30 22:00:51.627273 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:51.626951    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf" event={"ID":"9fdbd49b-36f2-4aeb-bc70-06f8647d2934","Type":"ContainerDied","Data":"d6b082aa6f2c69faeb7ddb721d31f5f0039b2532b8ffd6bbfe8746651da4d38f"}
Oct 30 22:00:52.633434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:52.633371    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf" event={"ID":"9fdbd49b-36f2-4aeb-bc70-06f8647d2934","Type":"ContainerStarted","Data":"604c4adc1f0b21d93ed499092e3e457e7adde4f0b8b8a58f093c485ab893a7d8"}
Oct 30 22:00:52.633911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:52.633472    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:00:52.787342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:00:52.787232    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf" podStartSLOduration=3.002631154 podStartE2EDuration="7.787218327s" totalImagesPullingTime="4.361883372s" totalInitContainerRuntime="422.703801ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:52.786887095 +0000 UTC m=+383.774695359" watchObservedRunningTime="2026-10-30 22:00:52.787218327 +0000 UTC m=+383.775026645"
Oct 30 22:01:03.639693 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:03.639666    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7fbfb566-jwxmf"
Oct 30 22:01:20.804724 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:01:20.804648    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:01:25.568859 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.568820    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-679b756675-lwmdb"]
Oct 30 22:01:25.574348 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.574325    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.660429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.660376    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49x9\" (UniqueName: \"kubernetes.io/projected/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-kube-api-access-c49x9\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.660595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.660440    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-tls-certs\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.660595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.660533    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-tmp\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.681648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.681618    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-679b756675-lwmdb"]
Oct 30 22:01:25.730664 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.730641    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 22:01:25.743971 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.743950    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dkbh2\""
Oct 30 22:01:25.744098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.744084    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 22:01:25.761279 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.761257    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49x9\" (UniqueName: \"kubernetes.io/projected/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-kube-api-access-c49x9\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.761361 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.761292    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-tls-certs\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.761361 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.761312    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-tmp\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.763730 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.763698    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-tmp\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.763836 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.763821    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-tls-certs\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.824310 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.824232    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49x9\" (UniqueName: \"kubernetes.io/projected/7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4-kube-api-access-c49x9\") pod \"kube-auth-proxy-679b756675-lwmdb\" (UID: \"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4\") " pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:25.885784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:25.885746    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb"
Oct 30 22:01:26.331661 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:26.331144    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-679b756675-lwmdb"]
Oct 30 22:01:26.333862 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:01:26.333837    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7591f97c_1ad1_4a4f_abfd_1cd5bcc121a4.slice/crio-3cb642f18c0f804122981f01f1b5433fef7fb2113af6c262e301e143990f63b0 WatchSource:0}: Error finding container 3cb642f18c0f804122981f01f1b5433fef7fb2113af6c262e301e143990f63b0: Status 404 returned error can't find the container with id 3cb642f18c0f804122981f01f1b5433fef7fb2113af6c262e301e143990f63b0
Oct 30 22:01:26.766622 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:26.766588    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb" event={"ID":"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4","Type":"ContainerStarted","Data":"3cb642f18c0f804122981f01f1b5433fef7fb2113af6c262e301e143990f63b0"}
Oct 30 22:01:29.305717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:29.305689    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 22:01:29.781547 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:29.781515    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb" event={"ID":"7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4","Type":"ContainerStarted","Data":"a06f5f16ff8963bd680713ece1afd85ea2da70986c61690c1702e1d9659463b0"}
Oct 30 22:01:29.901257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:29.901207    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-679b756675-lwmdb" podStartSLOduration=1.983239115 podStartE2EDuration="4.901193656s" totalImagesPullingTime="2.917954541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:29.900324302 +0000 UTC m=+420.888132565" watchObservedRunningTime="2026-10-30 22:01:29.901193656 +0000 UTC m=+420.889001919"
Oct 30 22:01:40.076206 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.076166    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"]
Oct 30 22:01:40.078537 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.078520    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.133072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.133028    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-htwjl\""
Oct 30 22:01:40.177771 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.177737    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.177963 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.177880    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzwp2\" (UniqueName: \"kubernetes.io/projected/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-kube-api-access-mzwp2\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.177963 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.177937    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.178113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.178065    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.181702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.181679    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 22:01:40.181830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.181739    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 22:01:40.279095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.279052    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.279288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.279101    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.279288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.279165    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzwp2\" (UniqueName: \"kubernetes.io/projected/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-kube-api-access-mzwp2\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.279288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.279202    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.279568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.279547    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.279636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.279598    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.281767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.281741    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 22:01:40.282969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.282952    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"]
Oct 30 22:01:40.289912 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.289889    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.342841 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.342780    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzwp2\" (UniqueName: \"kubernetes.io/projected/3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25-kube-api-access-mzwp2\") pod \"kserve-module-controller-manager-7566cf5fc8-jhpf7\" (UID: \"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.390039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.389985    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:40.822341 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:40.822239    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"]
Oct 30 22:01:40.825042 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:01:40.825010    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff5916a_58a7_4db2_8ff5_02ca4cdb6d25.slice/crio-87a3af5617ffdba84e7f843039a0511247d6c9c766815f0dcc1e8f3312f7be5c WatchSource:0}: Error finding container 87a3af5617ffdba84e7f843039a0511247d6c9c766815f0dcc1e8f3312f7be5c: Status 404 returned error can't find the container with id 87a3af5617ffdba84e7f843039a0511247d6c9c766815f0dcc1e8f3312f7be5c
Oct 30 22:01:41.829866 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:41.829829    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7" event={"ID":"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25","Type":"ContainerStarted","Data":"87a3af5617ffdba84e7f843039a0511247d6c9c766815f0dcc1e8f3312f7be5c"}
Oct 30 22:01:42.835274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:42.835240    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7" event={"ID":"3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25","Type":"ContainerStarted","Data":"7b35452509786b6b264d5b54c83d687d7e7145886bc72bdf67b6b5ec2e9905ee"}
Oct 30 22:01:42.835697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:42.835357    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:01:43.008802 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:43.008744    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7" podStartSLOduration=1.219025216 podStartE2EDuration="3.008724309s" totalImagesPullingTime="1.789699093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:43.007214093 +0000 UTC m=+433.995022368" watchObservedRunningTime="2026-10-30 22:01:43.008724309 +0000 UTC m=+433.996532573"
Oct 30 22:01:50.804507 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:01:50.804468    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:01:56.262071 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.262037    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:01:56.265343 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.265325    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.320055 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.320030    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-m79jn\""
Oct 30 22:01:56.368135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.368102    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 22:01:56.368135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.368102    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 22:01:56.368135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.368136    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 22:01:56.369343 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.369323    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:01:56.415586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.415555    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file\") pod \"limitador-limitador-74dfcfb894-fwjwj\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.415735 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.415675    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l7hx\" (UniqueName: \"kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx\") pod \"limitador-limitador-74dfcfb894-fwjwj\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.511135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.511103    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:01:56.516187 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.516144    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file\") pod \"limitador-limitador-74dfcfb894-fwjwj\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.516245 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.516217    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l7hx\" (UniqueName: \"kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx\") pod \"limitador-limitador-74dfcfb894-fwjwj\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.516807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.516787    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file\") pod \"limitador-limitador-74dfcfb894-fwjwj\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.578973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.578936    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l7hx\" (UniqueName: \"kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx\") pod \"limitador-limitador-74dfcfb894-fwjwj\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:56.877626 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:56.877538    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:01:57.268709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:57.268676    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:01:57.271589 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:01:57.271561    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c91a13_f2d0_4956_b2f7_c5843a56381d.slice/crio-43dd67ae96ad13a7aa7e2d62c8928a1ce5e6d5f96bc4aa840b359eb8731ae902 WatchSource:0}: Error finding container 43dd67ae96ad13a7aa7e2d62c8928a1ce5e6d5f96bc4aa840b359eb8731ae902: Status 404 returned error can't find the container with id 43dd67ae96ad13a7aa7e2d62c8928a1ce5e6d5f96bc4aa840b359eb8731ae902
Oct 30 22:01:57.893544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:01:57.893511    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj" event={"ID":"90c91a13-f2d0-4956-b2f7-c5843a56381d","Type":"ContainerStarted","Data":"43dd67ae96ad13a7aa7e2d62c8928a1ce5e6d5f96bc4aa840b359eb8731ae902"}
Oct 30 22:02:00.907549 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:00.907510    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj" event={"ID":"90c91a13-f2d0-4956-b2f7-c5843a56381d","Type":"ContainerStarted","Data":"8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4"}
Oct 30 22:02:00.907945 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:00.907602    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:02:01.027200 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:01.027083    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj" podStartSLOduration=1.536194591 podStartE2EDuration="5.02706974s" totalImagesPullingTime="3.490875149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:01.025849585 +0000 UTC m=+452.013657852" watchObservedRunningTime="2026-10-30 22:02:01.02706974 +0000 UTC m=+452.014878066"
Oct 30 22:02:03.840341 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:03.840315    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jhpf7"
Oct 30 22:02:11.912139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:11.912104    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:02:13.731347 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:13.731315    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:02:13.731785 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:13.731544    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj" podUID="90c91a13-f2d0-4956-b2f7-c5843a56381d" containerName="limitador" containerID="cri-o://8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4" gracePeriod=30
Oct 30 22:02:14.277566 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.277542    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:02:14.365942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.365851    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file\" (UniqueName: \"kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file\") pod \"90c91a13-f2d0-4956-b2f7-c5843a56381d\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") "
Oct 30 22:02:14.365942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.365912    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx\" (UniqueName: \"kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx\") pod \"90c91a13-f2d0-4956-b2f7-c5843a56381d\" (UID: \"90c91a13-f2d0-4956-b2f7-c5843a56381d\") "
Oct 30 22:02:14.366212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.366191    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file" pod "90c91a13-f2d0-4956-b2f7-c5843a56381d" (UID: "90c91a13-f2d0-4956-b2f7-c5843a56381d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:14.368254 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.368229    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx" pod "90c91a13-f2d0-4956-b2f7-c5843a56381d" (UID: "90c91a13-f2d0-4956-b2f7-c5843a56381d"). InnerVolumeSpecName "kube-api-access-8l7hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:14.467143 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.467104    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l7hx\" (UniqueName: \"kubernetes.io/projected/90c91a13-f2d0-4956-b2f7-c5843a56381d-kube-api-access-8l7hx\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.467143 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.467136    4536 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90c91a13-f2d0-4956-b2f7-c5843a56381d-config-file\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.968643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.968608    4536 generic.go:356] "Generic (PLEG): container finished" podID="90c91a13-f2d0-4956-b2f7-c5843a56381d" containerID="8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4" exitCode=0
Oct 30 22:02:14.969125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.968655    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"
Oct 30 22:02:14.969125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.968656    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj" event={"ID":"90c91a13-f2d0-4956-b2f7-c5843a56381d","Type":"ContainerDied","Data":"8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4"}
Oct 30 22:02:14.969125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.968709    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-fwjwj" event={"ID":"90c91a13-f2d0-4956-b2f7-c5843a56381d","Type":"ContainerDied","Data":"43dd67ae96ad13a7aa7e2d62c8928a1ce5e6d5f96bc4aa840b359eb8731ae902"}
Oct 30 22:02:14.969125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.968737    4536 scope.go:122] "RemoveContainer" containerID="8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4"
Oct 30 22:02:14.978366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.978345    4536 scope.go:122] "RemoveContainer" containerID="8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4"
Oct 30 22:02:14.978691 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:14.978675    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4\": container with ID starting with 8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4 not found: ID does not exist" containerID="8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4"
Oct 30 22:02:14.978762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:14.978700    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4"} err="failed to get container status \"8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4\": rpc error: code = NotFound desc = could not find container \"8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4\": container with ID starting with 8eb48937d32c88ac1dc16e5f60388e6a42e195c72cb2a6ede1f29d6789da73f4 not found: ID does not exist"
Oct 30 22:02:15.134453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:15.134420    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:02:15.184685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:15.184648    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-fwjwj"]
Oct 30 22:02:17.167548 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:17.167516    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90c91a13-f2d0-4956-b2f7-c5843a56381d" path="/var/lib/kubelet/pods/90c91a13-f2d0-4956-b2f7-c5843a56381d/volumes"
Oct 30 22:02:20.804594 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:20.804556    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:02:22.687125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.687086    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"]
Oct 30 22:02:22.687568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.687439    4536 cpu_manager.go:414] "containerMap: removing container" podUID="90c91a13-f2d0-4956-b2f7-c5843a56381d" containerName="limitador"
Oct 30 22:02:22.687568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.687452    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90c91a13-f2d0-4956-b2f7-c5843a56381d" containerName="limitador"
Oct 30 22:02:22.687568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.687506    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="90c91a13-f2d0-4956-b2f7-c5843a56381d" containerName="limitador"
Oct 30 22:02:22.690223 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.690203    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:22.729171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.729145    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/939c3d2e-da04-4bc6-a11a-bc3e008d14ce-data\") pod \"postgres-7cdbf6dc44-zbxqb\" (UID: \"939c3d2e-da04-4bc6-a11a-bc3e008d14ce\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:22.729324 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.729218    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvww\" (UniqueName: \"kubernetes.io/projected/939c3d2e-da04-4bc6-a11a-bc3e008d14ce-kube-api-access-zpvww\") pod \"postgres-7cdbf6dc44-zbxqb\" (UID: \"939c3d2e-da04-4bc6-a11a-bc3e008d14ce\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:22.752041 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.752016    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:02:22.752041 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.752017    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 22:02:22.794332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.794302    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:02:22.794332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.794301    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vp4dg\""
Oct 30 22:02:22.796280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.796259    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"]
Oct 30 22:02:22.830315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.830285    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvww\" (UniqueName: \"kubernetes.io/projected/939c3d2e-da04-4bc6-a11a-bc3e008d14ce-kube-api-access-zpvww\") pod \"postgres-7cdbf6dc44-zbxqb\" (UID: \"939c3d2e-da04-4bc6-a11a-bc3e008d14ce\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:22.830479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.830341    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/939c3d2e-da04-4bc6-a11a-bc3e008d14ce-data\") pod \"postgres-7cdbf6dc44-zbxqb\" (UID: \"939c3d2e-da04-4bc6-a11a-bc3e008d14ce\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:22.830662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.830645    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/939c3d2e-da04-4bc6-a11a-bc3e008d14ce-data\") pod \"postgres-7cdbf6dc44-zbxqb\" (UID: \"939c3d2e-da04-4bc6-a11a-bc3e008d14ce\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:22.893289 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:22.893262    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvww\" (UniqueName: \"kubernetes.io/projected/939c3d2e-da04-4bc6-a11a-bc3e008d14ce-kube-api-access-zpvww\") pod \"postgres-7cdbf6dc44-zbxqb\" (UID: \"939c3d2e-da04-4bc6-a11a-bc3e008d14ce\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:23.001349 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:23.001312    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:23.371867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:23.371844    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"]
Oct 30 22:02:23.374476 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:02:23.374449    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939c3d2e_da04_4bc6_a11a_bc3e008d14ce.slice/crio-6e06034a66149b64f68bf857e88fd59e0f041e6e454d61440019c6e31f8bb7b2 WatchSource:0}: Error finding container 6e06034a66149b64f68bf857e88fd59e0f041e6e454d61440019c6e31f8bb7b2: Status 404 returned error can't find the container with id 6e06034a66149b64f68bf857e88fd59e0f041e6e454d61440019c6e31f8bb7b2
Oct 30 22:02:24.007811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:24.007777    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb" event={"ID":"939c3d2e-da04-4bc6-a11a-bc3e008d14ce","Type":"ContainerStarted","Data":"6e06034a66149b64f68bf857e88fd59e0f041e6e454d61440019c6e31f8bb7b2"}
Oct 30 22:02:31.483890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:31.483861    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 22:02:32.048777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:32.048743    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb" event={"ID":"939c3d2e-da04-4bc6-a11a-bc3e008d14ce","Type":"ContainerStarted","Data":"73b8b4e8e6ee5494bce65cc2dd35fb5fc0bba489fb51ac3ca7fdbc7dd89cbe57"}
Oct 30 22:02:32.048979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:32.048962    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:32.174631 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:32.174559    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb" podStartSLOduration=2.118725983 podStartE2EDuration="10.174536717s" totalImagesPullingTime="8.055810734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:32.17312231 +0000 UTC m=+483.160930575" watchObservedRunningTime="2026-10-30 22:02:32.174536717 +0000 UTC m=+483.162344992"
Oct 30 22:02:38.081040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:38.081013    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-zbxqb"
Oct 30 22:02:42.371198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.371119    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"]
Oct 30 22:02:42.374256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.374235    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.477177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.477143    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"]
Oct 30 22:02:42.496605 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496576    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.496761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496620    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.496761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496640    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.496761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496698    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7c83fe-c33a-4512-9fb8-859c4eab9066-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.496879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496810    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.496879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496837    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h44hr\" (UniqueName: \"kubernetes.io/projected/8f7c83fe-c33a-4512-9fb8-859c4eab9066-kube-api-access-h44hr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.496879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.496867    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598120    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7c83fe-c33a-4512-9fb8-859c4eab9066-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598202    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598241    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h44hr\" (UniqueName: \"kubernetes.io/projected/8f7c83fe-c33a-4512-9fb8-859c4eab9066-kube-api-access-h44hr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598287    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598354    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598436    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.598983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.598468    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.602511 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.602147    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.604473 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.604212    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.604956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.604932    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7c83fe-c33a-4512-9fb8-859c4eab9066-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.605585 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.605558    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.605809 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.605791    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f7c83fe-c33a-4512-9fb8-859c4eab9066-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.666276 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.666210    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7c83fe-c33a-4512-9fb8-859c4eab9066-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.666450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.666384    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h44hr\" (UniqueName: \"kubernetes.io/projected/8f7c83fe-c33a-4512-9fb8-859c4eab9066-kube-api-access-h44hr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rzrlc\" (UID: \"8f7c83fe-c33a-4512-9fb8-859c4eab9066\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:42.685343 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:42.685309    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:43.113095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:43.113069    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"]
Oct 30 22:02:43.114737 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:02:43.114710    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7c83fe_c33a_4512_9fb8_859c4eab9066.slice/crio-3dbaf8ef6b7fe68c8f395649d917d46b132691af939a0de8ab9d6601b31b16c6 WatchSource:0}: Error finding container 3dbaf8ef6b7fe68c8f395649d917d46b132691af939a0de8ab9d6601b31b16c6: Status 404 returned error can't find the container with id 3dbaf8ef6b7fe68c8f395649d917d46b132691af939a0de8ab9d6601b31b16c6
Oct 30 22:02:43.116759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:43.116724    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:02:43.116834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:43.116785    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:02:44.098307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:44.098268    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc" event={"ID":"8f7c83fe-c33a-4512-9fb8-859c4eab9066","Type":"ContainerStarted","Data":"2ed57c787b26bb8d95cd3c8b236e25d9347d2ca9b7ca06c2fa7f0d1dd9452ea8"}
Oct 30 22:02:44.098794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:44.098321    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc" event={"ID":"8f7c83fe-c33a-4512-9fb8-859c4eab9066","Type":"ContainerStarted","Data":"3dbaf8ef6b7fe68c8f395649d917d46b132691af939a0de8ab9d6601b31b16c6"}
Oct 30 22:02:44.098794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:44.098555    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:44.100052 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:44.100024    4536 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-rzrlc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:44.100160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:44.100074    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc" podUID="8f7c83fe-c33a-4512-9fb8-859c4eab9066" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:44.326145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:44.326085    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc" podStartSLOduration=2.326067429 podStartE2EDuration="2.326067429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:44.281210516 +0000 UTC m=+495.269018780" watchObservedRunningTime="2026-10-30 22:02:44.326067429 +0000 UTC m=+495.313875692"
Oct 30 22:02:45.103030 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.103003    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rzrlc"
Oct 30 22:02:45.332057 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.332018    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"]
Oct 30 22:02:45.332280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.332257    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerName="discovery" containerID="cri-o://5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299" gracePeriod=30
Oct 30 22:02:45.418697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.418669    4536 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-n9mts container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.22:8080/ready\": dial tcp 10.133.0.22:8080: connect: connection refused" start-of-body=
Oct 30 22:02:45.418825 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.418723    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.22:8080/ready\": dial tcp 10.133.0.22:8080: connect: connection refused"
Oct 30 22:02:45.579421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.579375    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 22:02:45.625457 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625430    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.625628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625502    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.625628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625526    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.625628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625561    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.625628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625591    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.625628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625624    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.625899 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.625664    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts\") pod \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\" (UID: \"6b2efb62-29a0-4c63-b743-e9ded5ed22bb\") "
Oct 30 22:02:45.626099 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.626072    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:45.628020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.627992    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "kube-api-access-jjzwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:45.628148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.628136    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:45.628442 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.628388    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:45.628569 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.628482    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:45.628569 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.628492    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:45.628569 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.628514    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert" pod "6b2efb62-29a0-4c63-b743-e9ded5ed22bb" (UID: "6b2efb62-29a0-4c63-b743-e9ded5ed22bb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:45.726992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.726959    4536 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-cacerts\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.726992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.726988    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjzwk\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-kube-api-access-jjzwk\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.726992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.726999    4536 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-kubeconfig\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.727220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.727008    4536 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-local-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.727220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.727017    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.727220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.727026    4536 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-dns-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.727220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.727034    4536 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6b2efb62-29a0-4c63-b743-e9ded5ed22bb-istio-csr-ca-configmap\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:45.795574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.795533    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-687b4b7474-jbk42"]
Oct 30 22:02:45.795926 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.795909    4536 cpu_manager.go:414] "containerMap: removing container" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerName="discovery"
Oct 30 22:02:45.796005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.795928    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerName="discovery"
Oct 30 22:02:45.796060 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.796042    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerName="discovery"
Oct 30 22:02:45.799124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.799104    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:45.853140 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.853112    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:02:45.853329 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.853282    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:02:45.853329 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.853307    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fstvf\""
Oct 30 22:02:45.853436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.853384    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:02:45.862338 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.862309    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687b4b7474-jbk42"]
Oct 30 22:02:45.929485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.929447    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:45.929660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.929516    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bxf8\" (UniqueName: \"kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:45.929660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:45.929601    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.030359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.030277    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bxf8\" (UniqueName: \"kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.030359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.030328    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.030580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.030372    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.032974 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.032951    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.033079 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.032989    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.103269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.103236    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bxf8\" (UniqueName: \"kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8\") pod \"maas-controller-687b4b7474-jbk42\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") " pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.107635 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.107607    4536 generic.go:356] "Generic (PLEG): container finished" podID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" containerID="5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299" exitCode=0
Oct 30 22:02:46.107772 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.107650    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"
Oct 30 22:02:46.107772 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.107684    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" event={"ID":"6b2efb62-29a0-4c63-b743-e9ded5ed22bb","Type":"ContainerDied","Data":"5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299"}
Oct 30 22:02:46.107772 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.107724    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts" event={"ID":"6b2efb62-29a0-4c63-b743-e9ded5ed22bb","Type":"ContainerDied","Data":"d8131fd794eef8029504fd5f895c950a229f90daa0b49ba2cf613c744aac0503"}
Oct 30 22:02:46.107772 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.107743    4536 scope.go:122] "RemoveContainer" containerID="5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299"
Oct 30 22:02:46.111494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.111479    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:46.116759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.116743    4536 scope.go:122] "RemoveContainer" containerID="5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299"
Oct 30 22:02:46.117024 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:46.117003    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299\": container with ID starting with 5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299 not found: ID does not exist" containerID="5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299"
Oct 30 22:02:46.117070 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.117039    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299"} err="failed to get container status \"5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299\": rpc error: code = NotFound desc = could not find container \"5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299\": container with ID starting with 5bce8b91cf6ff5c17a38b3209b397ca5c2ad1a6fe30f961c184aa7bdfc297299 not found: ID does not exist"
Oct 30 22:02:46.279560 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.279531    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"]
Oct 30 22:02:46.283066 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.282998    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-n9mts"]
Oct 30 22:02:46.487358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:46.487059    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687b4b7474-jbk42"]
Oct 30 22:02:46.489883 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:02:46.489857    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517d5cf5_0ea2_4bb2_8aa7_2e56123b8b97.slice/crio-2a6c04c3ac97f8da2e6cbeea24af8dd768613dfdd62304ff6836452db4dcf9d8 WatchSource:0}: Error finding container 2a6c04c3ac97f8da2e6cbeea24af8dd768613dfdd62304ff6836452db4dcf9d8: Status 404 returned error can't find the container with id 2a6c04c3ac97f8da2e6cbeea24af8dd768613dfdd62304ff6836452db4dcf9d8
Oct 30 22:02:47.112465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:47.112431    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687b4b7474-jbk42" event={"ID":"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97","Type":"ContainerStarted","Data":"2a6c04c3ac97f8da2e6cbeea24af8dd768613dfdd62304ff6836452db4dcf9d8"}
Oct 30 22:02:47.162314 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:47.162279    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b2efb62-29a0-4c63-b743-e9ded5ed22bb" path="/var/lib/kubelet/pods/6b2efb62-29a0-4c63-b743-e9ded5ed22bb/volumes"
Oct 30 22:02:48.400795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.400759    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-cfgmp"]
Oct 30 22:02:48.407112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.407090    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.450759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.450725    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28aac9af-8737-449d-abd5-2de2057682c1-cert\") pod \"kserve-controller-manager-776cc66d47-cfgmp\" (UID: \"28aac9af-8737-449d-abd5-2de2057682c1\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.450954 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.450823    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c94h\" (UniqueName: \"kubernetes.io/projected/28aac9af-8737-449d-abd5-2de2057682c1-kube-api-access-4c94h\") pod \"kserve-controller-manager-776cc66d47-cfgmp\" (UID: \"28aac9af-8737-449d-abd5-2de2057682c1\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.510082 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.510043    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 22:02:48.510082 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.510042    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dnlqm\""
Oct 30 22:02:48.512843 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.512816    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-cfgmp"]
Oct 30 22:02:48.514002 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.513980    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"]
Oct 30 22:02:48.519829 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.519805    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.551903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.551863    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c94h\" (UniqueName: \"kubernetes.io/projected/28aac9af-8737-449d-abd5-2de2057682c1-kube-api-access-4c94h\") pod \"kserve-controller-manager-776cc66d47-cfgmp\" (UID: \"28aac9af-8737-449d-abd5-2de2057682c1\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.552072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.551932    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28aac9af-8737-449d-abd5-2de2057682c1-cert\") pod \"kserve-controller-manager-776cc66d47-cfgmp\" (UID: \"28aac9af-8737-449d-abd5-2de2057682c1\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.554595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.554570    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28aac9af-8737-449d-abd5-2de2057682c1-cert\") pod \"kserve-controller-manager-776cc66d47-cfgmp\" (UID: \"28aac9af-8737-449d-abd5-2de2057682c1\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.576639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.576610    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"]
Oct 30 22:02:48.576961 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.576937    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 30 22:02:48.577076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.577059    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vjstp\""
Oct 30 22:02:48.615990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.615959    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c94h\" (UniqueName: \"kubernetes.io/projected/28aac9af-8737-449d-abd5-2de2057682c1-kube-api-access-4c94h\") pod \"kserve-controller-manager-776cc66d47-cfgmp\" (UID: \"28aac9af-8737-449d-abd5-2de2057682c1\") " pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.652470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.652353    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-cmdm2\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.652637 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.652523    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wswfq\" (UniqueName: \"kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq\") pod \"llmisvc-controller-manager-869d5dcc5d-cmdm2\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.723879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.723845    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:48.753561 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.753523    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-cmdm2\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.753722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.753600    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wswfq\" (UniqueName: \"kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq\") pod \"llmisvc-controller-manager-869d5dcc5d-cmdm2\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.756871 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.756844    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-cmdm2\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.818346 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.818316    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wswfq\" (UniqueName: \"kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq\") pod \"llmisvc-controller-manager-869d5dcc5d-cmdm2\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:48.835088 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:48.835055    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:49.107456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:49.107424    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-cfgmp"]
Oct 30 22:02:49.110640 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:02:49.110613    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28aac9af_8737_449d_abd5_2de2057682c1.slice/crio-e2700a235682134a85dc0357e01ee1ba500542e36c524d18c7f2d844fa41f7af WatchSource:0}: Error finding container e2700a235682134a85dc0357e01ee1ba500542e36c524d18c7f2d844fa41f7af: Status 404 returned error can't find the container with id e2700a235682134a85dc0357e01ee1ba500542e36c524d18c7f2d844fa41f7af
Oct 30 22:02:49.124113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:49.124077    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp" event={"ID":"28aac9af-8737-449d-abd5-2de2057682c1","Type":"ContainerStarted","Data":"e2700a235682134a85dc0357e01ee1ba500542e36c524d18c7f2d844fa41f7af"}
Oct 30 22:02:49.216335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:49.216299    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"]
Oct 30 22:02:49.219237 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:02:49.219192    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4acf06d3_9f3f_4cd8_a9bd_9032c29397dd.slice/crio-894b3b814dd0459c644f5bce6b5459a45b707884fc47689688675ea8b3ae6514 WatchSource:0}: Error finding container 894b3b814dd0459c644f5bce6b5459a45b707884fc47689688675ea8b3ae6514: Status 404 returned error can't find the container with id 894b3b814dd0459c644f5bce6b5459a45b707884fc47689688675ea8b3ae6514
Oct 30 22:02:50.131157 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:50.131124    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2" event={"ID":"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd","Type":"ContainerStarted","Data":"894b3b814dd0459c644f5bce6b5459a45b707884fc47689688675ea8b3ae6514"}
Oct 30 22:02:50.132831 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:50.132799    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687b4b7474-jbk42" event={"ID":"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97","Type":"ContainerStarted","Data":"e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e"}
Oct 30 22:02:50.133024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:50.133006    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:02:50.261415 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:50.261331    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-687b4b7474-jbk42" podStartSLOduration=2.267601367 podStartE2EDuration="5.261315432s" totalImagesPullingTime="2.993714065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:50.26078713 +0000 UTC m=+501.248595396" watchObservedRunningTime="2026-10-30 22:02:50.261315432 +0000 UTC m=+501.249123689"
Oct 30 22:02:50.807383 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:50.807347    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:02:52.144991 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:52.144916    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2" event={"ID":"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd","Type":"ContainerStarted","Data":"1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a"}
Oct 30 22:02:52.145360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:52.145144    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:02:52.263189 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:52.263135    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2" podStartSLOduration=1.7820439129999999 podStartE2EDuration="4.263102638s" totalImagesPullingTime="2.481058725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:52.262614335 +0000 UTC m=+503.250422599" watchObservedRunningTime="2026-10-30 22:02:52.263102638 +0000 UTC m=+503.250910901"
Oct 30 22:02:53.031467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.031434    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"]
Oct 30 22:02:53.038721 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.038686    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.096352 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.096317    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6q29\" (UniqueName: \"kubernetes.io/projected/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-kube-api-access-t6q29\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.096352 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.096362    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.096621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.096489    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.096670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.096655    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.143520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.143489    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 22:02:53.143520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.143489    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 22:02:53.143746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.143555    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 22:02:53.143746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.143604    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-7hbvz\""
Oct 30 22:02:53.145026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.145002    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"]
Oct 30 22:02:53.150234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.150209    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp" event={"ID":"28aac9af-8737-449d-abd5-2de2057682c1","Type":"ContainerStarted","Data":"6488610f004b45d8202e81b03d4fd681f5b9e6f2c316cc4ce7954aa23a6fa702"}
Oct 30 22:02:53.197726 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.197693    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.197918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.197767    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6q29\" (UniqueName: \"kubernetes.io/projected/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-kube-api-access-t6q29\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.197918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.197797    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.197918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.197829    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.197918 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:53.197860    4536 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 22:02:53.197918 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:53.197897    4536 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 22:02:53.198091 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:53.197927    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-tls-cert podName:cf440c32-3c0d-4220-a26e-f66f3aed1d9c nodeName:}" failed. No retries permitted until 2026-10-30 22:02:53.697907328 +0000 UTC m=+504.685715571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-h7wbr" (UID: "cf440c32-3c0d-4220-a26e-f66f3aed1d9c") : secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 22:02:53.198091 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:02:53.197945    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-proxy-source podName:cf440c32-3c0d-4220-a26e-f66f3aed1d9c nodeName:}" failed. No retries permitted until 2026-10-30 22:02:53.697936902 +0000 UTC m=+504.685745143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-h7wbr" (UID: "cf440c32-3c0d-4220-a26e-f66f3aed1d9c") : configmap "usage-tenancy-proxy-source" not found
Oct 30 22:02:53.198512 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.198496    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.260918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.260889    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6q29\" (UniqueName: \"kubernetes.io/projected/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-kube-api-access-t6q29\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.279091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.279038    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp" podStartSLOduration=2.061345621 podStartE2EDuration="5.279022998s" totalImagesPullingTime="3.217677377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:53.27751806 +0000 UTC m=+504.265326325" watchObservedRunningTime="2026-10-30 22:02:53.279022998 +0000 UTC m=+504.266831261"
Oct 30 22:02:53.703149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.703109    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.703334 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.703164    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.704017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.703987    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.705839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.705815    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf440c32-3c0d-4220-a26e-f66f3aed1d9c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-h7wbr\" (UID: \"cf440c32-3c0d-4220-a26e-f66f3aed1d9c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:53.951520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:53.951479    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:02:54.154869 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:54.154839    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:02:54.327108 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:54.327036    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"]
Oct 30 22:02:54.330331 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:02:54.330290    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf440c32_3c0d_4220_a26e_f66f3aed1d9c.slice/crio-3e8553ce05d8b4c1d2290994acc55f794db626dfdc6be9d44609e314da732eac WatchSource:0}: Error finding container 3e8553ce05d8b4c1d2290994acc55f794db626dfdc6be9d44609e314da732eac: Status 404 returned error can't find the container with id 3e8553ce05d8b4c1d2290994acc55f794db626dfdc6be9d44609e314da732eac
Oct 30 22:02:55.162236 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:02:55.162201    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr" event={"ID":"cf440c32-3c0d-4220-a26e-f66f3aed1d9c","Type":"ContainerStarted","Data":"3e8553ce05d8b4c1d2290994acc55f794db626dfdc6be9d44609e314da732eac"}
Oct 30 22:03:01.140625 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:01.140577    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:03:10.230672 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:10.230635    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr" event={"ID":"cf440c32-3c0d-4220-a26e-f66f3aed1d9c","Type":"ContainerStarted","Data":"107a0bc4dc1639b3824e96e760f01b7f2b2139eeeaaef6dbf56f100164140958"}
Oct 30 22:03:10.231081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:10.230718    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:03:10.350334 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:10.350262    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr" podStartSLOduration=3.32876619 podStartE2EDuration="18.350246922s" totalImagesPullingTime="15.021480732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:10.349085293 +0000 UTC m=+521.336893557" watchObservedRunningTime="2026-10-30 22:03:10.350246922 +0000 UTC m=+521.338055185"
Oct 30 22:03:20.804615 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:03:20.804581    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:03:21.241537 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:21.241508    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-h7wbr"
Oct 30 22:03:23.152914 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:23.152881    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:03:24.737670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.737633    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"]
Oct 30 22:03:24.800361 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.800326    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.801358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.801334    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"]
Oct 30 22:03:24.862137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.862107    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxbhl\" (UniqueName: \"kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.862137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.862141    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.862379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.862265    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.862379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.862371    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.908488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.908464    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zj8wn\""
Oct 30 22:03:24.908488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.908465    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 22:03:24.908488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.908474    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 22:03:24.962806 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.962773    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.962959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.962830    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.962959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.962857    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxbhl\" (UniqueName: \"kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.962959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.962876    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.963465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.963443    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.965363 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.965345    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:24.965471 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:24.965417    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:25.025284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:25.025215    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxbhl\" (UniqueName: \"kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl\") pod \"maas-api-6d669c7fcd-wjngx\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:25.132252 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:25.132215    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:25.168036 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:25.168014    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-cfgmp"
Oct 30 22:03:25.517703 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:25.517670    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"]
Oct 30 22:03:25.520083 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:03:25.520046    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0295692a_f4a4_4035_a6ec_f1b6db013152.slice/crio-e1b38faf30044d97be26b28bb40804c4b18cf37e00dcaaafa2715b371d7279f3 WatchSource:0}: Error finding container e1b38faf30044d97be26b28bb40804c4b18cf37e00dcaaafa2715b371d7279f3: Status 404 returned error can't find the container with id e1b38faf30044d97be26b28bb40804c4b18cf37e00dcaaafa2715b371d7279f3
Oct 30 22:03:26.297455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:26.297423    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx" event={"ID":"0295692a-f4a4-4035-a6ec-f1b6db013152","Type":"ContainerStarted","Data":"e1b38faf30044d97be26b28bb40804c4b18cf37e00dcaaafa2715b371d7279f3"}
Oct 30 22:03:28.308717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:28.308684    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx" event={"ID":"0295692a-f4a4-4035-a6ec-f1b6db013152","Type":"ContainerStarted","Data":"d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03"}
Oct 30 22:03:28.309186 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:28.308812    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:28.428459 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:28.428391    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx" podStartSLOduration=2.301442308 podStartE2EDuration="4.428377898s" totalImagesPullingTime="2.12693559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:28.427240557 +0000 UTC m=+539.415048821" watchObservedRunningTime="2026-10-30 22:03:28.428377898 +0000 UTC m=+539.416186170"
Oct 30 22:03:34.317786 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:34.317755    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:35.390259 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.390223    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"]
Oct 30 22:03:35.393269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.393246    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.443358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.443332    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.443358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.443357    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8j7t\" (UniqueName: \"kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.443532 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.443419    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.443532 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.443498    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.497281 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.497245    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"]
Oct 30 22:03:35.544202 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.544167    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.544388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.544206    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8j7t\" (UniqueName: \"kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.544388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.544262    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.544388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.544286    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.545011 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.544984    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.546979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.546953    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.547163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.547042    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.609016 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.608981    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8j7t\" (UniqueName: \"kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t\") pod \"maas-api-54d58dc998-8k2cl\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") " pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:35.705164 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:35.705134    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:36.083170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:36.083144    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"]
Oct 30 22:03:36.085851 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:03:36.085824    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0d67b8_579b_446b_b8cd_803547e31e55.slice/crio-cfece5cc18263b01d34ba082085fd08cd493273500087463281aaf0cf567af91 WatchSource:0}: Error finding container cfece5cc18263b01d34ba082085fd08cd493273500087463281aaf0cf567af91: Status 404 returned error can't find the container with id cfece5cc18263b01d34ba082085fd08cd493273500087463281aaf0cf567af91
Oct 30 22:03:36.342455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:36.342361    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl" event={"ID":"0a0d67b8-579b-446b-b8cd-803547e31e55","Type":"ContainerStarted","Data":"cfece5cc18263b01d34ba082085fd08cd493273500087463281aaf0cf567af91"}
Oct 30 22:03:37.354968 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:37.354923    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl" event={"ID":"0a0d67b8-579b-446b-b8cd-803547e31e55","Type":"ContainerStarted","Data":"91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed"}
Oct 30 22:03:37.355435 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:37.355083    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:37.503374 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:37.503319    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl" podStartSLOduration=1.407957291 podStartE2EDuration="2.503304295s" totalImagesPullingTime="1.095347004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:37.502539853 +0000 UTC m=+548.490348118" watchObservedRunningTime="2026-10-30 22:03:37.503304295 +0000 UTC m=+548.491112557"
Oct 30 22:03:43.364683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.364657    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:03:43.537185 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.537154    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"]
Oct 30 22:03:43.537447 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.537419    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx" podUID="0295692a-f4a4-4035-a6ec-f1b6db013152" containerName="maas-api" containerID="cri-o://d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03" gracePeriod=30
Oct 30 22:03:43.812520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.812494    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:43.918036 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.917955    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls\") pod \"0295692a-f4a4-4035-a6ec-f1b6db013152\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") "
Oct 30 22:03:43.918036 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.918029    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl\" (UniqueName: \"kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl\") pod \"0295692a-f4a4-4035-a6ec-f1b6db013152\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") "
Oct 30 22:03:43.918264 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.918105    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca\") pod \"0295692a-f4a4-4035-a6ec-f1b6db013152\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") "
Oct 30 22:03:43.918264 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.918152    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs\") pod \"0295692a-f4a4-4035-a6ec-f1b6db013152\" (UID: \"0295692a-f4a4-4035-a6ec-f1b6db013152\") "
Oct 30 22:03:43.918527 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.918501    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca" pod "0295692a-f4a4-4035-a6ec-f1b6db013152" (UID: "0295692a-f4a4-4035-a6ec-f1b6db013152"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:43.920469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.920440    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl" pod "0295692a-f4a4-4035-a6ec-f1b6db013152" (UID: "0295692a-f4a4-4035-a6ec-f1b6db013152"). InnerVolumeSpecName "kube-api-access-lxbhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:43.920469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.920457    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls" pod "0295692a-f4a4-4035-a6ec-f1b6db013152" (UID: "0295692a-f4a4-4035-a6ec-f1b6db013152"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:43.920624 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:43.920473    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs" pod "0295692a-f4a4-4035-a6ec-f1b6db013152" (UID: "0295692a-f4a4-4035-a6ec-f1b6db013152"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:44.019209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.019179    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:44.019209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.019205    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxbhl\" (UniqueName: \"kubernetes.io/projected/0295692a-f4a4-4035-a6ec-f1b6db013152-kube-api-access-lxbhl\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:44.019209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.019215    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0295692a-f4a4-4035-a6ec-f1b6db013152-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:44.019465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.019224    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0295692a-f4a4-4035-a6ec-f1b6db013152-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:44.386278 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.386238    4536 generic.go:356] "Generic (PLEG): container finished" podID="0295692a-f4a4-4035-a6ec-f1b6db013152" containerID="d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03" exitCode=0
Oct 30 22:03:44.386795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.386286    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"
Oct 30 22:03:44.386795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.386325    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx" event={"ID":"0295692a-f4a4-4035-a6ec-f1b6db013152","Type":"ContainerDied","Data":"d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03"}
Oct 30 22:03:44.386795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.386382    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx" event={"ID":"0295692a-f4a4-4035-a6ec-f1b6db013152","Type":"ContainerDied","Data":"e1b38faf30044d97be26b28bb40804c4b18cf37e00dcaaafa2715b371d7279f3"}
Oct 30 22:03:44.386795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.386430    4536 scope.go:122] "RemoveContainer" containerID="d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03"
Oct 30 22:03:44.395700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.395681    4536 scope.go:122] "RemoveContainer" containerID="d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03"
Oct 30 22:03:44.395965 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:03:44.395943    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03\": container with ID starting with d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03 not found: ID does not exist" containerID="d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03"
Oct 30 22:03:44.396024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.395978    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03"} err="failed to get container status \"d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03\": rpc error: code = NotFound desc = could not find container \"d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03\": container with ID starting with d790c029aed509333ff23cc46c7f1cbe7981a728a6c6b18b57f8063f188e5c03 not found: ID does not exist"
Oct 30 22:03:44.555007 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.554974    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"]
Oct 30 22:03:44.604014 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:44.603983    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-wjngx"]
Oct 30 22:03:45.163890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:03:45.163861    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0295692a-f4a4-4035-a6ec-f1b6db013152" path="/var/lib/kubelet/pods/0295692a-f4a4-4035-a6ec-f1b6db013152/volumes"
Oct 30 22:03:50.804571 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:03:50.804460    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:04:05.576241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.576201    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"]
Oct 30 22:04:05.576676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.576596    4536 cpu_manager.go:414] "containerMap: removing container" podUID="0295692a-f4a4-4035-a6ec-f1b6db013152" containerName="maas-api"
Oct 30 22:04:05.576676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.576608    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0295692a-f4a4-4035-a6ec-f1b6db013152" containerName="maas-api"
Oct 30 22:04:05.576676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.576664    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="0295692a-f4a4-4035-a6ec-f1b6db013152" containerName="maas-api"
Oct 30 22:04:05.578758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.578743    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.639096 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.639063    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"]
Oct 30 22:04:05.711416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.711364    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99xk\" (UniqueName: \"kubernetes.io/projected/acd884c1-1fa7-4900-8bbc-acead715fb0e-kube-api-access-d99xk\") pod \"llmisvc-controller-manager-6466c554d8-ntktz\" (UID: \"acd884c1-1fa7-4900-8bbc-acead715fb0e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.711586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.711482    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acd884c1-1fa7-4900-8bbc-acead715fb0e-cert\") pod \"llmisvc-controller-manager-6466c554d8-ntktz\" (UID: \"acd884c1-1fa7-4900-8bbc-acead715fb0e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.812027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.811989    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99xk\" (UniqueName: \"kubernetes.io/projected/acd884c1-1fa7-4900-8bbc-acead715fb0e-kube-api-access-d99xk\") pod \"llmisvc-controller-manager-6466c554d8-ntktz\" (UID: \"acd884c1-1fa7-4900-8bbc-acead715fb0e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.812193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.812047    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acd884c1-1fa7-4900-8bbc-acead715fb0e-cert\") pod \"llmisvc-controller-manager-6466c554d8-ntktz\" (UID: \"acd884c1-1fa7-4900-8bbc-acead715fb0e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.814737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.814708    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acd884c1-1fa7-4900-8bbc-acead715fb0e-cert\") pod \"llmisvc-controller-manager-6466c554d8-ntktz\" (UID: \"acd884c1-1fa7-4900-8bbc-acead715fb0e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.876182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.876112    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99xk\" (UniqueName: \"kubernetes.io/projected/acd884c1-1fa7-4900-8bbc-acead715fb0e-kube-api-access-d99xk\") pod \"llmisvc-controller-manager-6466c554d8-ntktz\" (UID: \"acd884c1-1fa7-4900-8bbc-acead715fb0e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:05.890585 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:05.890564    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:06.261583 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:06.261549    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"]
Oct 30 22:04:06.264511 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:04:06.264477    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd884c1_1fa7_4900_8bbc_acead715fb0e.slice/crio-319ead6f0301ed7a23718e11c1dd0090bbfc25cfccc39b3a9dfc829c0beafd33 WatchSource:0}: Error finding container 319ead6f0301ed7a23718e11c1dd0090bbfc25cfccc39b3a9dfc829c0beafd33: Status 404 returned error can't find the container with id 319ead6f0301ed7a23718e11c1dd0090bbfc25cfccc39b3a9dfc829c0beafd33
Oct 30 22:04:06.481798 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:06.481762    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz" event={"ID":"acd884c1-1fa7-4900-8bbc-acead715fb0e","Type":"ContainerStarted","Data":"319ead6f0301ed7a23718e11c1dd0090bbfc25cfccc39b3a9dfc829c0beafd33"}
Oct 30 22:04:07.487365 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:07.487331    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz" event={"ID":"acd884c1-1fa7-4900-8bbc-acead715fb0e","Type":"ContainerStarted","Data":"cb3681a01813aa0f2d56a6b4ea79287902a484709030bd227af1f00658c102cf"}
Oct 30 22:04:07.487756 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:07.487431    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:07.605762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:07.605711    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz" podStartSLOduration=1.897759363 podStartE2EDuration="2.60569599s" totalImagesPullingTime="707.936627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:07.604563754 +0000 UTC m=+578.592372005" watchObservedRunningTime="2026-10-30 22:04:07.60569599 +0000 UTC m=+578.593504253"
Oct 30 22:04:20.805091 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:04:20.805008    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:04:38.493951 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.493922    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-ntktz"
Oct 30 22:04:38.663742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.663712    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"]
Oct 30 22:04:38.663934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.663913    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2" podUID="4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" containerName="manager" containerID="cri-o://1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a" gracePeriod=30
Oct 30 22:04:38.905671 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.905648    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:04:38.993036 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.993006    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq\" (UniqueName: \"kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq\") pod \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") "
Oct 30 22:04:38.993185 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.993057    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert\" (UniqueName: \"kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert\") pod \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\" (UID: \"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd\") "
Oct 30 22:04:38.995377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.995351    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert" pod "4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" (UID: "4acf06d3-9f3f-4cd8-a9bd-9032c29397dd"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:04:38.995501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:38.995473    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq" pod "4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" (UID: "4acf06d3-9f3f-4cd8-a9bd-9032c29397dd"). InnerVolumeSpecName "kube-api-access-wswfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:39.094323 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.094240    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wswfq\" (UniqueName: \"kubernetes.io/projected/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-kube-api-access-wswfq\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:39.094323 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.094272    4536 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:39.623354 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.623322    4536 generic.go:356] "Generic (PLEG): container finished" podID="4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" containerID="1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a" exitCode=0
Oct 30 22:04:39.623872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.623419    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"
Oct 30 22:04:39.623872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.623437    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2" event={"ID":"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd","Type":"ContainerDied","Data":"1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a"}
Oct 30 22:04:39.623872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.623490    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2" event={"ID":"4acf06d3-9f3f-4cd8-a9bd-9032c29397dd","Type":"ContainerDied","Data":"894b3b814dd0459c644f5bce6b5459a45b707884fc47689688675ea8b3ae6514"}
Oct 30 22:04:39.623872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.623512    4536 scope.go:122] "RemoveContainer" containerID="1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a"
Oct 30 22:04:39.631739 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.631716    4536 scope.go:122] "RemoveContainer" containerID="1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a"
Oct 30 22:04:39.631989 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:04:39.631968    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a\": container with ID starting with 1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a not found: ID does not exist" containerID="1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a"
Oct 30 22:04:39.632052 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.632002    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a"} err="failed to get container status \"1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a\": rpc error: code = NotFound desc = could not find container \"1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a\": container with ID starting with 1b7982e4bb844c750f039cbc4da153b46fe511ac7ee937b949cfcc199bbc701a not found: ID does not exist"
Oct 30 22:04:39.789756 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.789724    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"]
Oct 30 22:04:39.839476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:39.839443    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-cmdm2"]
Oct 30 22:04:41.162822 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:04:41.162784    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" path="/var/lib/kubelet/pods/4acf06d3-9f3f-4cd8-a9bd-9032c29397dd/volumes"
Oct 30 22:04:50.805228 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:04:50.805195    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:05:15.398618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.398583    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"]
Oct 30 22:05:15.399291 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.399130    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" containerName="manager"
Oct 30 22:05:15.399291 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.399150    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" containerName="manager"
Oct 30 22:05:15.399291 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.399242    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acf06d3-9f3f-4cd8-a9bd-9032c29397dd" containerName="manager"
Oct 30 22:05:15.402199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.402179    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405272    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405328    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405425    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405454    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405530 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405493    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405594    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405651    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qvq2\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405733    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.405795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.405781    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506261 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506225    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"]
Oct 30 22:05:15.506573 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506549    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506614 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506595    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506615    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506661    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506691    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506806 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506710    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qvq2\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506806 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506748    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506806 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506787    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506814    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.506979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.506962    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.507093 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.507032    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.507458 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.507426    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.507577 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.507427    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.507577 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.507563    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.509372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.509343    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.509666 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.509619    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.545714 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.545679    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9qrfj\""
Oct 30 22:05:15.572781 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.572744    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.576237 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.576212    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qvq2\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2\") pod \"maas-default-gateway-openshift-default-76597d6855-28k55\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:15.714864 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:15.714833    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:16.167726 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:16.167698    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"]
Oct 30 22:05:16.170343 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:05:16.170314    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ff9e68_3c3e_46c2_ac8d_21fd5f4c4282.slice/crio-adf80efed1926bbdf61a8951c8bd8d3a563599ed60c65417a4ac8dd6398c50ae WatchSource:0}: Error finding container adf80efed1926bbdf61a8951c8bd8d3a563599ed60c65417a4ac8dd6398c50ae: Status 404 returned error can't find the container with id adf80efed1926bbdf61a8951c8bd8d3a563599ed60c65417a4ac8dd6398c50ae
Oct 30 22:05:16.172357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:16.172342    4536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:05:16.793146 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:16.793111    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" event={"ID":"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282","Type":"ContainerStarted","Data":"adf80efed1926bbdf61a8951c8bd8d3a563599ed60c65417a4ac8dd6398c50ae"}
Oct 30 22:05:19.994555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:19.994524    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:05:19.994815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:19.994592    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:05:19.994815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:19.994621    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:05:20.813537 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:20.813496    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" event={"ID":"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282","Type":"ContainerStarted","Data":"8f9262ddfddfb8b8cf6a78eebfaef9a626d2f0f58a41c29e3d7026c2db6ec73b"}
Oct 30 22:05:20.821080 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:05:20.821049    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:05:21.041998 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:21.041946    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podStartSLOduration=2.220837714 podStartE2EDuration="6.04193171s" totalImagesPullingTime="3.821093996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:20.990945298 +0000 UTC m=+651.978753564" watchObservedRunningTime="2026-10-30 22:05:21.04193171 +0000 UTC m=+652.029740052"
Oct 30 22:05:21.715317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:21.715281    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:21.716869 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:21.716848    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:05:21.716931 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:21.716907    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:05:22.715580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:22.715543    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:05:22.715986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:22.715617    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:05:23.715418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:23.715360    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:05:23.715599 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:23.715454    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:05:24.715903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:24.715861    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:05:24.716382 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:24.715925    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:05:25.716007 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:25.715973    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 22:05:25.716419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:25.716024    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 22:05:27.716486 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:27.716439    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:27.717013 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:27.716520    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:28.717630 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:28.717597    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-28k55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:28.718042 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:28.717663    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:28.718931 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:28.718907    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:28.719168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:28.719148    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:28.719909 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:05:28.719889    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:05:50.805300 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:05:50.805218    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:06:20.805430 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:06:20.805377    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:06:50.804294 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:06:50.804258    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:07:08.953470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:08.953432    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"]
Oct 30 22:07:08.957034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:08.957018    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.010578 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.010549    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-c76d9e-serving-cert\""
Oct 30 22:07:09.028809 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.028784    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.028990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.028816    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdzrm\" (UniqueName: \"kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.028990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.028859    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.028990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.028974    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.061153 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.061130    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"]
Oct 30 22:07:09.129600 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.129568    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdzrm\" (UniqueName: \"kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.129782 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.129637    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.129782 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.129667    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.129782 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.129728    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.129938 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:09.129783    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-serving-cert: secret "maas-api-e2e-worker-w6-c76d9e-serving-cert" not found
Oct 30 22:07:09.129938 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:09.129872    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls podName:c08065b3-fe54-489e-aef6-07855e1e382f nodeName:}" failed. No retries permitted until 2026-10-30 22:07:09.629849989 +0000 UTC m=+760.617658235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls") pod "maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" (UID: "c08065b3-fe54-489e-aef6-07855e1e382f") : secret "maas-api-e2e-worker-w6-c76d9e-serving-cert" not found
Oct 30 22:07:09.130426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.130388    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.132166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.132146    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.191899 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.191847    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdzrm\" (UniqueName: \"kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.633819 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.633786    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.636208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.636186    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:09.869704 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:09.869667    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:10.242064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:10.242039    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"]
Oct 30 22:07:10.245122 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:07:10.245097    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc08065b3_fe54_489e_aef6_07855e1e382f.slice/crio-0b8e00a09b9068b6c9c0a02ae50095c5bd093a82ed3c5c73c7eb1716981d16d3 WatchSource:0}: Error finding container 0b8e00a09b9068b6c9c0a02ae50095c5bd093a82ed3c5c73c7eb1716981d16d3: Status 404 returned error can't find the container with id 0b8e00a09b9068b6c9c0a02ae50095c5bd093a82ed3c5c73c7eb1716981d16d3
Oct 30 22:07:10.288846 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:10.288815    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" event={"ID":"c08065b3-fe54-489e-aef6-07855e1e382f","Type":"ContainerStarted","Data":"0b8e00a09b9068b6c9c0a02ae50095c5bd093a82ed3c5c73c7eb1716981d16d3"}
Oct 30 22:07:12.323288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:12.323241    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" event={"ID":"c08065b3-fe54-489e-aef6-07855e1e382f","Type":"ContainerStarted","Data":"84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c"}
Oct 30 22:07:12.325328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:12.324254    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:12.447524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:12.447468    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" podStartSLOduration=3.460606714 podStartE2EDuration="4.447452488s" totalImagesPullingTime="986.845774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:12.446375145 +0000 UTC m=+763.434183411" watchObservedRunningTime="2026-10-30 22:07:12.447452488 +0000 UTC m=+763.435260751"
Oct 30 22:07:19.339171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.339142    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:07:19.752469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.752437    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"]
Oct 30 22:07:19.755807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.755791    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.809458 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.809419    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-075cbb-serving-cert\""
Oct 30 22:07:19.816804 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.816772    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"]
Oct 30 22:07:19.831258 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.831232    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.831390 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.831273    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.831390 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.831295    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.831494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.831445    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96599\" (UniqueName: \"kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.932771 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.932735    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.932962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.932799    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.932962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.932828    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.932962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.932859    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96599\" (UniqueName: \"kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.932962 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:19.932907    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-serving-cert: secret "maas-api-e2e-worker-w5-075cbb-serving-cert" not found
Oct 30 22:07:19.933182 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:19.932969    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls podName:a8737966-b199-4970-baa9-922ee6f0b015 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:20.432948781 +0000 UTC m=+771.420757038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls") pod "maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" (UID: "a8737966-b199-4970-baa9-922ee6f0b015") : secret "maas-api-e2e-worker-w5-075cbb-serving-cert" not found
Oct 30 22:07:19.933539 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.933520    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.935302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.935276    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:19.999292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:19.999260    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96599\" (UniqueName: \"kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:20.438216 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:20.438181    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:20.440740 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:20.440717    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\") pod \"maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:20.668339 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:20.668306    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:20.806217 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:20.806183    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:07:21.047459 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:21.047433    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"]
Oct 30 22:07:21.050111 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:07:21.050075    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8737966_b199_4970_baa9_922ee6f0b015.slice/crio-7939a5554d2496c3290042b7a4d797f139f94bb7ebadcbb60c582f8c30a07d39 WatchSource:0}: Error finding container 7939a5554d2496c3290042b7a4d797f139f94bb7ebadcbb60c582f8c30a07d39: Status 404 returned error can't find the container with id 7939a5554d2496c3290042b7a4d797f139f94bb7ebadcbb60c582f8c30a07d39
Oct 30 22:07:21.367314 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:21.367229    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" event={"ID":"a8737966-b199-4970-baa9-922ee6f0b015","Type":"ContainerStarted","Data":"7939a5554d2496c3290042b7a4d797f139f94bb7ebadcbb60c582f8c30a07d39"}
Oct 30 22:07:22.373080 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:22.373049    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" event={"ID":"a8737966-b199-4970-baa9-922ee6f0b015","Type":"ContainerStarted","Data":"9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c"}
Oct 30 22:07:22.373510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:22.373148    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:22.493465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:22.493412    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" podStartSLOduration=2.684179799 podStartE2EDuration="3.493377995s" totalImagesPullingTime="809.198196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:22.491817318 +0000 UTC m=+773.479625582" watchObservedRunningTime="2026-10-30 22:07:22.493377995 +0000 UTC m=+773.481186257"
Oct 30 22:07:28.384667 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:28.384634    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:07:30.784956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.784922    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"]
Oct 30 22:07:30.789534 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.789513    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:30.863097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.863062    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"]
Oct 30 22:07:30.909964 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.909930    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-13b8d8-serving-cert\""
Oct 30 22:07:30.951972 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.951932    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:30.952135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.951996    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:30.952135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.952090    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:30.952135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:30.952121    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjgzv\" (UniqueName: \"kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.052882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.052787    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.052882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.052829    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjgzv\" (UniqueName: \"kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.052882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.052866    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.053195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.052903    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.053195 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:31.053007    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-serving-cert: secret "maas-api-e2e-worker-w2-13b8d8-serving-cert" not found
Oct 30 22:07:31.053195 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:31.053065    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls podName:6380a3ed-6530-41d7-919d-e0e759f62513 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:31.553047725 +0000 UTC m=+782.540855972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls") pod "maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" (UID: "6380a3ed-6530-41d7-919d-e0e759f62513") : secret "maas-api-e2e-worker-w2-13b8d8-serving-cert" not found
Oct 30 22:07:31.053670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.053651    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.055529 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.055506    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.115508 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.115478    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjgzv\" (UniqueName: \"kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.557439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.557386    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.559984 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.559954    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\") pod \"maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:31.702446 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:31.702413    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:32.077421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:32.077352    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"]
Oct 30 22:07:32.083307 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:07:32.083274    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6380a3ed_6530_41d7_919d_e0e759f62513.slice/crio-53c29917dd4382ed4bfd1a8bd47d9b6ce6c7fbac25749d820703327bac783167 WatchSource:0}: Error finding container 53c29917dd4382ed4bfd1a8bd47d9b6ce6c7fbac25749d820703327bac783167: Status 404 returned error can't find the container with id 53c29917dd4382ed4bfd1a8bd47d9b6ce6c7fbac25749d820703327bac783167
Oct 30 22:07:32.433501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:32.433414    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" event={"ID":"6380a3ed-6530-41d7-919d-e0e759f62513","Type":"ContainerStarted","Data":"53c29917dd4382ed4bfd1a8bd47d9b6ce6c7fbac25749d820703327bac783167"}
Oct 30 22:07:33.440189 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:33.440101    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" event={"ID":"6380a3ed-6530-41d7-919d-e0e759f62513","Type":"ContainerStarted","Data":"04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c"}
Oct 30 22:07:33.440725 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:33.440243    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:33.610737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:33.610682    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" podStartSLOduration=2.741651064 podStartE2EDuration="3.610668298s" totalImagesPullingTime="869.017234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:33.560038985 +0000 UTC m=+784.547847261" watchObservedRunningTime="2026-10-30 22:07:33.610668298 +0000 UTC m=+784.598476562"
Oct 30 22:07:35.952452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:35.952417    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"]
Oct 30 22:07:35.956042 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:35.956023    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:35.996710 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:35.996680    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:35.996875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:35.996713    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltz75\" (UniqueName: \"kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:35.996875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:35.996814    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:35.996875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:35.996862    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.009605 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.009578    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-2a4123-serving-cert\""
Oct 30 22:07:36.060330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.060290    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"]
Oct 30 22:07:36.098223 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.098191    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.098417 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.098239    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltz75\" (UniqueName: \"kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.098417 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.098321    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.098417 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.098358    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.098582 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:36.098519    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-serving-cert: secret "maas-api-e2e-worker-w0-2a4123-serving-cert" not found
Oct 30 22:07:36.098641 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:36.098598    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls podName:87f414d1-0414-4945-8552-b2ac20bf4649 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:36.59857896 +0000 UTC m=+787.586387204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls") pod "maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" (UID: "87f414d1-0414-4945-8552-b2ac20bf4649") : secret "maas-api-e2e-worker-w0-2a4123-serving-cert" not found
Oct 30 22:07:36.099202 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.099174    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.101324 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.101295    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.162152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.162118    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltz75\" (UniqueName: \"kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.603695 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.603661    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.606233 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.606201    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:36.868257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:36.868160    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:37.247562 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:37.247529    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"]
Oct 30 22:07:37.250512 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:07:37.250472    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f414d1_0414_4945_8552_b2ac20bf4649.slice/crio-9b3fc9453d3a58e8c073304133b6e1d5949873651abf027526be43d3349f584d WatchSource:0}: Error finding container 9b3fc9453d3a58e8c073304133b6e1d5949873651abf027526be43d3349f584d: Status 404 returned error can't find the container with id 9b3fc9453d3a58e8c073304133b6e1d5949873651abf027526be43d3349f584d
Oct 30 22:07:37.458723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:37.458689    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" event={"ID":"87f414d1-0414-4945-8552-b2ac20bf4649","Type":"ContainerStarted","Data":"9b3fc9453d3a58e8c073304133b6e1d5949873651abf027526be43d3349f584d"}
Oct 30 22:07:38.466042 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:38.466003    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" event={"ID":"87f414d1-0414-4945-8552-b2ac20bf4649","Type":"ContainerStarted","Data":"d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d"}
Oct 30 22:07:38.466466 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:38.466115    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:38.636122 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:38.636067    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" podStartSLOduration=2.808682701 podStartE2EDuration="3.63604695s" totalImagesPullingTime="827.364249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:38.585377421 +0000 UTC m=+789.573185685" watchObservedRunningTime="2026-10-30 22:07:38.63604695 +0000 UTC m=+789.623855215"
Oct 30 22:07:39.452725 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:39.452689    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:07:44.478984 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:44.478959    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:07:50.804992 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:50.804953    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:07:51.654933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.654896    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"]
Oct 30 22:07:51.658761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.658740    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.745935 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.745902    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.745935 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.745944    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.746212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.745965    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.746212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.746135    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdvg\" (UniqueName: \"kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.761637 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.761611    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-97aa64-serving-cert\""
Oct 30 22:07:51.763322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.763301    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"]
Oct 30 22:07:51.846620 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.846586    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.847083 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.846645    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdvg\" (UniqueName: \"kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.847083 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.846706    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.847083 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.846729    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.847083 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:51.846745    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-serving-cert: secret "maas-api-e2e-worker-w4-97aa64-serving-cert" not found
Oct 30 22:07:51.847083 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:07:51.846819    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls podName:1b3d1f77-b03e-4553-9331-75bffb1d391a nodeName:}" failed. No retries permitted until 2026-10-30 22:07:52.346803895 +0000 UTC m=+803.334612140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls") pod "maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" (UID: "1b3d1f77-b03e-4553-9331-75bffb1d391a") : secret "maas-api-e2e-worker-w4-97aa64-serving-cert" not found
Oct 30 22:07:51.847529 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.847507    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.849166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.849148    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:51.912444 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:51.912328    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdvg\" (UniqueName: \"kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:52.352129 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:52.352095    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:52.354642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:52.354618    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\") pod \"maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:52.571884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:52.571847    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:52.953110 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:52.952998    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"]
Oct 30 22:07:52.955802 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:07:52.955772    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3d1f77_b03e_4553_9331_75bffb1d391a.slice/crio-ca4720f1ab2bb076e101260949f830a726d0f23d54a5491cfb48b53ec9dc48bf WatchSource:0}: Error finding container ca4720f1ab2bb076e101260949f830a726d0f23d54a5491cfb48b53ec9dc48bf: Status 404 returned error can't find the container with id ca4720f1ab2bb076e101260949f830a726d0f23d54a5491cfb48b53ec9dc48bf
Oct 30 22:07:53.544283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:53.544242    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" event={"ID":"1b3d1f77-b03e-4553-9331-75bffb1d391a","Type":"ContainerStarted","Data":"ca4720f1ab2bb076e101260949f830a726d0f23d54a5491cfb48b53ec9dc48bf"}
Oct 30 22:07:54.550696 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:54.550660    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" event={"ID":"1b3d1f77-b03e-4553-9331-75bffb1d391a","Type":"ContainerStarted","Data":"0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856"}
Oct 30 22:07:54.551104 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:54.550753    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:07:54.720967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:07:54.720911    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" podStartSLOduration=2.888906529 podStartE2EDuration="3.720897362s" totalImagesPullingTime="831.990833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:54.670476014 +0000 UTC m=+805.658284277" watchObservedRunningTime="2026-10-30 22:07:54.720897362 +0000 UTC m=+805.708705624"
Oct 30 22:08:00.560618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:00.560577    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:08:20.805158 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:08:20.805120    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:08:50.805978 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:08:50.805884    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:08:51.278426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.278384    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"]
Oct 30 22:08:51.281903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.281888    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.384238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.384209    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-13b8d8\"/\"simulator-w2-13b8d8-kserve-self-signed-certs\""
Oct 30 22:08:51.384387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.384244    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-13b8d8\"/\"default-dockercfg-9vlnw\""
Oct 30 22:08:51.384694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.384679    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-13b8d8\"/\"openshift-service-ca.crt\""
Oct 30 22:08:51.384741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.384712    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-13b8d8\"/\"kube-root-ca.crt\""
Oct 30 22:08:51.386791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.386776    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"]
Oct 30 22:08:51.416037 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.416003    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.416436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.416371    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.416613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.416456    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.416613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.416485    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhfxc\" (UniqueName: \"kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.416613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.416550    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.416732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.416638    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518025 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.517993    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518191 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518048    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518191 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518067    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518191    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhfxc\" (UniqueName: \"kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518237    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518284    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518518 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518442    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518649 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518629    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.518778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.518753    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.520699 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.520667    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.520924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.520907    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.581270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.581198    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhfxc\" (UniqueName: \"kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc\") pod \"simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") " pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.610212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.610166    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:08:51.993780 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:51.993758    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"]
Oct 30 22:08:51.996225 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:08:51.996197    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod081b695c_a7d3_4519_b0af_745bfcfa2ad1.slice/crio-beb547f771783cf1004df0d76b686aca9a6c44e366b10e2451706a9a3e1242cf WatchSource:0}: Error finding container beb547f771783cf1004df0d76b686aca9a6c44e366b10e2451706a9a3e1242cf: Status 404 returned error can't find the container with id beb547f771783cf1004df0d76b686aca9a6c44e366b10e2451706a9a3e1242cf
Oct 30 22:08:52.806595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:52.806512    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" event={"ID":"081b695c-a7d3-4519-b0af-745bfcfa2ad1","Type":"ContainerStarted","Data":"beb547f771783cf1004df0d76b686aca9a6c44e366b10e2451706a9a3e1242cf"}
Oct 30 22:08:53.812483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:53.812382    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" event={"ID":"081b695c-a7d3-4519-b0af-745bfcfa2ad1","Type":"ContainerStarted","Data":"4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf"}
Oct 30 22:08:53.930849 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:53.930794    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podStartSLOduration=1.440982076 podStartE2EDuration="2.930781581s" totalImagesPullingTime="1.489799505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:53.929784353 +0000 UTC m=+864.917592618" watchObservedRunningTime="2026-10-30 22:08:53.930781581 +0000 UTC m=+864.918589847"
Oct 30 22:08:54.610695 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:08:54.610656    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:09:01.610761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:01.610727    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:09:01.623417 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:01.623371    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:09:01.861128 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:01.861040    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:09:02.632494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.632458    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"]
Oct 30 22:09:02.636148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.636132    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.717455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.717418    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.717636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.717526    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.717636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.717590    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kllgt\" (UniqueName: \"kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.717636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.717609    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.717636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.717628    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.717815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.717741    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.739074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.739042    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2a4123\"/\"default-dockercfg-snr45\""
Oct 30 22:09:02.739249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.739091    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2a4123\"/\"simulator-w0-2a4123-kserve-self-signed-certs\""
Oct 30 22:09:02.739249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.739162    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2a4123\"/\"kube-root-ca.crt\""
Oct 30 22:09:02.739249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.739173    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2a4123\"/\"openshift-service-ca.crt\""
Oct 30 22:09:02.741285 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.741268    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"]
Oct 30 22:09:02.818307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818275    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818325    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818365    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kllgt\" (UniqueName: \"kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818381    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818469    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818790 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818569    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818790 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818734    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818817    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.818904 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.818882    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.820853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.820834    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.821016 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.820999    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.884390 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.884308    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kllgt\" (UniqueName: \"kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt\") pod \"simulator-w0-2a4123-kserve-55fdf675dd-thxzg\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") " pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:02.948469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:02.948426    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:03.327374 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:03.327353    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"]
Oct 30 22:09:03.329171 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:09:03.329148    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5799f173_0882_4895_8049_087dfac11054.slice/crio-cbd18f243cb4fe7944e05d8a85bf79dbc0955719504f5b476e1b0af83bad6c65 WatchSource:0}: Error finding container cbd18f243cb4fe7944e05d8a85bf79dbc0955719504f5b476e1b0af83bad6c65: Status 404 returned error can't find the container with id cbd18f243cb4fe7944e05d8a85bf79dbc0955719504f5b476e1b0af83bad6c65
Oct 30 22:09:03.862281 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:03.862248    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" event={"ID":"5799f173-0882-4895-8049-087dfac11054","Type":"ContainerStarted","Data":"7f8834e931b5687c10122fa3b1b54117ff07eaf8ecf94d5bd32ba6e8ab461467"}
Oct 30 22:09:03.862281 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:03.862289    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" event={"ID":"5799f173-0882-4895-8049-087dfac11054","Type":"ContainerStarted","Data":"cbd18f243cb4fe7944e05d8a85bf79dbc0955719504f5b476e1b0af83bad6c65"}
Oct 30 22:09:03.948563 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:03.948531    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:03.986251 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:03.986204    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podStartSLOduration=1.986189212 podStartE2EDuration="1.986189212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:03.983566657 +0000 UTC m=+874.971374945" watchObservedRunningTime="2026-10-30 22:09:03.986189212 +0000 UTC m=+874.973997475"
Oct 30 22:09:12.948928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:12.948900    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:12.961297 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:12.961276    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:13.919590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:13.919557    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:09:14.972286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:14.972252    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"]
Oct 30 22:09:14.975807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:14.975792    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025119    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025172    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025200    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025241    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025296    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025384    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025455    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025483    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.025683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.025536    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckrj2\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.031453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.031434    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-c76d9e-openshift-default-dockercfg-r2lmn\""
Oct 30 22:09:15.082384 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.082355    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"]
Oct 30 22:09:15.126808 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126771    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.126808 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126807    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126846    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126867    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126912    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126941    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.126978    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127009    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckrj2\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127113    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127210    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127247    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127277    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127641    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.127736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.127701    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.129528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.129507    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.129623 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.129585    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.193228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.193188    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckrj2\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.193591 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.193572    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.289983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.289890    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:15.722437 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:09:15.722386    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c61b4c_c214_4eb0_acbe_6515a319d684.slice/crio-c85cbb0e75734f63d927698a1e8546b2a0e4c794ad84d85fb005259faac1b1d0 WatchSource:0}: Error finding container c85cbb0e75734f63d927698a1e8546b2a0e4c794ad84d85fb005259faac1b1d0: Status 404 returned error can't find the container with id c85cbb0e75734f63d927698a1e8546b2a0e4c794ad84d85fb005259faac1b1d0
Oct 30 22:09:15.722925 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.722903    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"]
Oct 30 22:09:15.724817 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.724788    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:09:15.724890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.724850    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:09:15.724890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.724879    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:09:15.919415 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.919377    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" event={"ID":"a5c61b4c-c214-4eb0-acbe-6515a319d684","Type":"ContainerStarted","Data":"976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3"}
Oct 30 22:09:15.919587 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:15.919444    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" event={"ID":"a5c61b4c-c214-4eb0-acbe-6515a319d684","Type":"ContainerStarted","Data":"c85cbb0e75734f63d927698a1e8546b2a0e4c794ad84d85fb005259faac1b1d0"}
Oct 30 22:09:16.093694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:16.093587    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podStartSLOduration=2.093571804 podStartE2EDuration="2.093571804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:16.042712284 +0000 UTC m=+887.030520560" watchObservedRunningTime="2026-10-30 22:09:16.093571804 +0000 UTC m=+887.081380067"
Oct 30 22:09:16.293420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:16.292777    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:16.296583 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:16.296539    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:09:16.296744 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:16.296631    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:09:17.291811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:17.291772    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:09:17.292782 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:17.292367    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:09:18.296550 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:18.296505    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:09:18.296933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:18.296566    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:09:19.292422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:19.291842    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:09:19.292422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:19.291907    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:09:20.291419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:20.290751    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:09:20.291419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:20.290809    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:09:20.820152 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:09:20.820109    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:09:21.294423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:21.291837    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:09:21.294423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:21.291903    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:09:22.410472 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:22.410440    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:22.411359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:22.411200    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:22.412489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:09:22.412468    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:09:50.806063 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:09:50.806020    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:10:20.805512 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:10:20.805425    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:10:21.066436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.066323    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"]
Oct 30 22:10:21.070035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.070010    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.180155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.180130    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-13b8d8\"/\"premium-w2-13b8d8-kserve-self-signed-certs\""
Oct 30 22:10:21.182173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.182148    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"]
Oct 30 22:10:21.215836 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.215794    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzsvl\" (UniqueName: \"kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.216147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.215865    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.216147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.215907    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.216147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.215959    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.216147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.216017    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.216147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.216051    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317203    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317237    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317299    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzsvl\" (UniqueName: \"kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317342    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317430    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317506    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317819 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317678    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.317885 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.317860    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.318093 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.318062    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.319910 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.319891    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.320195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.320172    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.390573 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.390540    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzsvl\" (UniqueName: \"kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl\") pod \"premium-w2-13b8d8-kserve-5fddc4866-xdhhv\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") " pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:21.682665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:21.682581    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:22.072008 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:22.071984    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"]
Oct 30 22:10:22.074108 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:10:22.074080    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf448fdd2_77cc_4c38_ba05_1078286a2a32.slice/crio-b5aba5947cadebcbc7148f37212e0402978a047c157f0b4850e97ad9f5ea5907 WatchSource:0}: Error finding container b5aba5947cadebcbc7148f37212e0402978a047c157f0b4850e97ad9f5ea5907: Status 404 returned error can't find the container with id b5aba5947cadebcbc7148f37212e0402978a047c157f0b4850e97ad9f5ea5907
Oct 30 22:10:22.244892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:22.244860    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" event={"ID":"f448fdd2-77cc-4c38-ba05-1078286a2a32","Type":"ContainerStarted","Data":"4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019"}
Oct 30 22:10:22.245051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:22.244901    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" event={"ID":"f448fdd2-77cc-4c38-ba05-1078286a2a32","Type":"ContainerStarted","Data":"b5aba5947cadebcbc7148f37212e0402978a047c157f0b4850e97ad9f5ea5907"}
Oct 30 22:10:22.367006 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:22.366900    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podStartSLOduration=1.366886694 podStartE2EDuration="1.366886694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:22.364817911 +0000 UTC m=+953.352626184" watchObservedRunningTime="2026-10-30 22:10:22.366886694 +0000 UTC m=+953.354694956"
Oct 30 22:10:22.683460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:22.683376    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:31.682734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:31.682691    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:31.695665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:31.695641    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:32.298836 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:32.298807    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:10:50.803873 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:10:50.803835    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:10:52.895247 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:52.895210    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"]
Oct 30 22:10:52.899161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:52.899139    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:52.952827 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:52.952800    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-97aa64\"/\"openshift-service-ca.crt\""
Oct 30 22:10:53.001888 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.001847    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-97aa64\"/\"kube-root-ca.crt\""
Oct 30 22:10:53.002056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.001914    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-97aa64\"/\"premium-w4-97aa64-kserve-self-signed-certs\""
Oct 30 22:10:53.002056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.001935    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-97aa64\"/\"default-dockercfg-v57fc\""
Oct 30 22:10:53.004320 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.004302    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"]
Oct 30 22:10:53.044245 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.044202    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.050093 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.050057    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.050583 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.050532    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.050896 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.050869    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.051226 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.050916    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.051226 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.051016    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxpmc\" (UniqueName: \"kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.151760 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.151666    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxpmc\" (UniqueName: \"kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.151760 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.151749    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.151806    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.151858    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.151916    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.151947    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152263 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.152130    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152263 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.152205    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.152505 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.152478    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.154506 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.154480    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.154787 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.154772    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.214475 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.214441    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxpmc\" (UniqueName: \"kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc\") pod \"premium-w4-97aa64-kserve-6565d769b6-f8vzr\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") " pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.511238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.511210    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:10:53.887735 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:53.887708    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"]
Oct 30 22:10:53.889774 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:10:53.889748    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b5bbc31_dd7e_4d5b_8b5b_22c20ac95ffb.slice/crio-d3ef4d5888687f54d96b4f8b12989c4c7dd677e729c05b71b63f0f2a55946169 WatchSource:0}: Error finding container d3ef4d5888687f54d96b4f8b12989c4c7dd677e729c05b71b63f0f2a55946169: Status 404 returned error can't find the container with id d3ef4d5888687f54d96b4f8b12989c4c7dd677e729c05b71b63f0f2a55946169
Oct 30 22:10:54.391160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:54.391117    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" event={"ID":"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb","Type":"ContainerStarted","Data":"9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1"}
Oct 30 22:10:54.391160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:54.391163    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" event={"ID":"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb","Type":"ContainerStarted","Data":"d3ef4d5888687f54d96b4f8b12989c4c7dd677e729c05b71b63f0f2a55946169"}
Oct 30 22:10:54.509900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:54.509841    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podStartSLOduration=2.509824685 podStartE2EDuration="2.509824685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:54.50949473 +0000 UTC m=+985.497303018" watchObservedRunningTime="2026-10-30 22:10:54.509824685 +0000 UTC m=+985.497632948"
Oct 30 22:10:54.511562 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:10:54.511544    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:11:03.512123 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:11:03.512087    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:11:03.524521 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:11:03.524500    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:11:04.459804 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:11:04.459778    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:11:20.805544 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:11:20.805507    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:11:50.804534 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:11:50.804456    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:12:16.702671 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.702635    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"]
Oct 30 22:12:16.706072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.706052    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.760847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.760813    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-075cbb-openshift-default-dockercfg-88pqw\""
Oct 30 22:12:16.808118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808084    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808235    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808273    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808315    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808478 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808458    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808514 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808490    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808567 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808527    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808567 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808549    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.808677 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.808593    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6tnp\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.811510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.811488    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"]
Oct 30 22:12:16.909310 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909268    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909348    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909373    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909449    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909474    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909501    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6tnp\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909558    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909648    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.909805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.909680    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.910113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.910082    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.910270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.910083    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.910344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.910157    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.910344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.910164    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.910529 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.910374    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.912340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.912309    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.912813 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.912793    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.972183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.972106    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:16.972350 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:16.972196    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6tnp\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:17.019952 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.019910    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:17.454293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.454261    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"]
Oct 30 22:12:17.454944 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:12:17.454911    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b1de6f_91a5_4948_abe0_e96e29cb68e5.slice/crio-b9fd78deef6d88ceba976bf78f582adc596a5f176c8dd6c45b797beaaa29a0a8 WatchSource:0}: Error finding container b9fd78deef6d88ceba976bf78f582adc596a5f176c8dd6c45b797beaaa29a0a8: Status 404 returned error can't find the container with id b9fd78deef6d88ceba976bf78f582adc596a5f176c8dd6c45b797beaaa29a0a8
Oct 30 22:12:17.457882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.457840    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:12:17.457986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.457928    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:12:17.457986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.457970    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:12:17.809061 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.808972    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" event={"ID":"a5b1de6f-91a5-4948-abe0-e96e29cb68e5","Type":"ContainerStarted","Data":"3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb"}
Oct 30 22:12:17.809061 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.809041    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" event={"ID":"a5b1de6f-91a5-4948-abe0-e96e29cb68e5","Type":"ContainerStarted","Data":"b9fd78deef6d88ceba976bf78f582adc596a5f176c8dd6c45b797beaaa29a0a8"}
Oct 30 22:12:17.936762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:17.936700    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podStartSLOduration=1.936680959 podStartE2EDuration="1.936680959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:17.934563322 +0000 UTC m=+1068.922371584" watchObservedRunningTime="2026-10-30 22:12:17.936680959 +0000 UTC m=+1068.924489222"
Oct 30 22:12:18.020757 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:18.020712    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:18.024675 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:18.024634    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 22:12:18.024861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:18.024711    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 22:12:19.020919 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:19.020884    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 22:12:19.021358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:19.020938    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 22:12:20.020871 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:20.020834    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 22:12:20.021038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:20.020889    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 22:12:20.819943 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:12:20.819906    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:12:21.020834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:21.020795    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 22:12:21.021046 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:21.020854    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 22:12:22.020746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:22.020715    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 22:12:22.020999 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:22.020773    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 22:12:23.020911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:23.020881    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 22:12:23.021292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:23.020944    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 22:12:25.006279 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:25.006240    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:25.006764 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:25.006654    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:25.007673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:25.007655    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:12:47.324807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.324777    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"]
Oct 30 22:12:47.328128 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.328111    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.430703 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.430662    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-13b8d8-openshift-default-dockercfg-8442c\""
Oct 30 22:12:47.432926 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.432883    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"]
Oct 30 22:12:47.453580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.453546    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.453729 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.453593    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.453794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.453771    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.453843 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.453814    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.453954 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.453932    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.454003 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.453971    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.454050 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.454000    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.454096 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.454066    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.454096 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.454081    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5l46\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.554841 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.554806    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.554846    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.554901    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.554935    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.554970    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555027    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555313 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555049    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5l46\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555313 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555094    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555313 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555135    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555315    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555454    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555492    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555570    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.555791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.555774    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.557515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.557495    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.557664 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.557646    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.617702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.617625    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.617853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.617786    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5l46\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:47.641585 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:47.641556    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:48.070441 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:48.070376    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"]
Oct 30 22:12:48.072193 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:12:48.072165    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda281ebb8_b31d_48f6_a61e_99c710ebab88.slice/crio-3b47c5b6403cc58ed72639ae9b7435a753350b8c66c921f7ea01e91ee248dbcf WatchSource:0}: Error finding container 3b47c5b6403cc58ed72639ae9b7435a753350b8c66c921f7ea01e91ee248dbcf: Status 404 returned error can't find the container with id 3b47c5b6403cc58ed72639ae9b7435a753350b8c66c921f7ea01e91ee248dbcf
Oct 30 22:12:48.074560 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:48.074530    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:12:48.074627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:48.074595    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:12:48.074627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:48.074624    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:12:48.971452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:48.971376    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" event={"ID":"a281ebb8-b31d-48f6-a61e-99c710ebab88","Type":"ContainerStarted","Data":"fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b"}
Oct 30 22:12:48.971956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:48.971505    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" event={"ID":"a281ebb8-b31d-48f6-a61e-99c710ebab88","Type":"ContainerStarted","Data":"3b47c5b6403cc58ed72639ae9b7435a753350b8c66c921f7ea01e91ee248dbcf"}
Oct 30 22:12:49.152727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:49.152671    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podStartSLOduration=2.152656569 podStartE2EDuration="2.152656569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:49.149986561 +0000 UTC m=+1100.137794827" watchObservedRunningTime="2026-10-30 22:12:49.152656569 +0000 UTC m=+1100.140464907"
Oct 30 22:12:49.641926 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:49.641892    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:49.643257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:49.643228    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 22:12:49.643425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:49.643296    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 22:12:50.642467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:50.642387    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 22:12:50.642959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:50.642495    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 22:12:50.821551 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:12:50.821519    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:12:51.642328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:51.642295    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 22:12:51.642538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:51.642356    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 22:12:52.642209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:52.642176    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 22:12:52.642379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:52.642245    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 22:12:53.642051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:53.642008    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 22:12:53.642714 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:53.642094    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 22:12:54.877661 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:54.877631    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:54.882665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:54.882640    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:12:54.883365 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:12:54.883346    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:13:02.827503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.827469    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"]
Oct 30 22:13:02.831382 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.831361    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.885304 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.885277    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-97aa64-openshift-default-dockercfg-dhxrl\""
Oct 30 22:13:02.937893 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.937857    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"]
Oct 30 22:13:02.945942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.945911    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.945975    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946063    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946095    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shbqv\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946130    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946264    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946348 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946336    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946370    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:02.946426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:02.946418    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047735 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047688    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047735 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047740    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047764    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047815    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047876    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047915    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047943    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shbqv\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.047984    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.048379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.048017    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.048379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.048291    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.048379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.048365    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.048643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.048622    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.048796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.048780    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.048914 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.048893    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.050524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.050500    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.050627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.050564    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.124033 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.123963    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.124033 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.123964    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shbqv\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.145877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.145852    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:03.588967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.588935    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"]
Oct 30 22:13:03.620453 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:13:03.620426    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9a5a29_7dcf_45c7_a03b_8ae4fc906005.slice/crio-a246a2a2ba0c0d2625d5ebaf213f2c54fb89d24e8748e1870476f34f61f56a4c WatchSource:0}: Error finding container a246a2a2ba0c0d2625d5ebaf213f2c54fb89d24e8748e1870476f34f61f56a4c: Status 404 returned error can't find the container with id a246a2a2ba0c0d2625d5ebaf213f2c54fb89d24e8748e1870476f34f61f56a4c
Oct 30 22:13:03.622847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.622812    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:13:03.622937 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.622894    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:13:03.622937 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:03.622924    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:13:04.073144 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:04.073105    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" event={"ID":"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005","Type":"ContainerStarted","Data":"af02960b9e3685cd58c7f57855804265f67230866c4fedd0bf64304c537d2b87"}
Oct 30 22:13:04.073594 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:04.073165    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" event={"ID":"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005","Type":"ContainerStarted","Data":"a246a2a2ba0c0d2625d5ebaf213f2c54fb89d24e8748e1870476f34f61f56a4c"}
Oct 30 22:13:04.146409 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:04.146378    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:04.147690 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:04.147667    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:13:04.147826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:04.147711    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:13:04.199604 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:04.199543    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podStartSLOduration=2.199520971 podStartE2EDuration="2.199520971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:04.195522428 +0000 UTC m=+1115.183330703" watchObservedRunningTime="2026-10-30 22:13:04.199520971 +0000 UTC m=+1115.187329235"
Oct 30 22:13:05.147105 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:05.147070    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:13:05.147576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:05.147127    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:13:06.146423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:06.146368    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:13:06.146626 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:06.146442    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:13:07.147300 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:07.147259    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:13:07.147723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:07.147317    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:13:08.146445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:08.146412    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:13:08.146633 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:08.146476    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:13:09.146972 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:09.146937    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:13:09.147381 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:09.146996    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:13:10.321144 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:10.320914    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:10.321827 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:10.321793    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:10.323072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:10.323051    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:13:20.805569 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:13:20.805536    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:13:32.675364 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.675327    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"]
Oct 30 22:13:32.679317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.679255    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.737072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.737041    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"]
Oct 30 22:13:32.792211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.792175    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-075cbb\"/\"kube-root-ca.crt\""
Oct 30 22:13:32.792377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.792219    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-075cbb\"/\"openshift-service-ca.crt\""
Oct 30 22:13:32.796002 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.795973    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.796121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796088    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.796121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796111    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgwgw\" (UniqueName: \"kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.796317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796292    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.796456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796333    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.796456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796428    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-075cbb\"/\"default-dockercfg-dzqtz\""
Oct 30 22:13:32.796574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796472    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-075cbb\"/\"distinct-w5-075cbb-kserve-self-signed-certs\""
Oct 30 22:13:32.796574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.796491    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.897674 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.897624    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.897867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.897694    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.897867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.897741    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.897867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.897764    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgwgw\" (UniqueName: \"kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.897867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.897845    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.898100 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.897870    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.898165 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.898128    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.898229 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.898195    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.898299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.898276    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.900049 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.900029    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.900492 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.900471    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.967839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.967810    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgwgw\" (UniqueName: \"kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw\") pod \"distinct-w5-075cbb-kserve-69ff9b8d5-drlw9\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") " pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:32.991655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:32.991629    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:33.428879 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:13:33.428853    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddd044c2_e404_4b13_9c32_067038dc37a6.slice/crio-1049c1939fedbca82f7744b36365b2f97903c011f88987ed66f5d8d03b5e1011 WatchSource:0}: Error finding container 1049c1939fedbca82f7744b36365b2f97903c011f88987ed66f5d8d03b5e1011: Status 404 returned error can't find the container with id 1049c1939fedbca82f7744b36365b2f97903c011f88987ed66f5d8d03b5e1011
Oct 30 22:13:33.434001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:33.433982    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"]
Oct 30 22:13:34.244131 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:34.244097    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" event={"ID":"ddd044c2-e404-4b13-9c32-067038dc37a6","Type":"ContainerStarted","Data":"8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92"}
Oct 30 22:13:34.244566 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:34.244140    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" event={"ID":"ddd044c2-e404-4b13-9c32-067038dc37a6","Type":"ContainerStarted","Data":"1049c1939fedbca82f7744b36365b2f97903c011f88987ed66f5d8d03b5e1011"}
Oct 30 22:13:34.379425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:34.379363    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podStartSLOduration=2.379348567 podStartE2EDuration="2.379348567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:34.37752422 +0000 UTC m=+1145.365332484" watchObservedRunningTime="2026-10-30 22:13:34.379348567 +0000 UTC m=+1145.367156863"
Oct 30 22:13:34.992344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:34.992314    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:42.992120 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:42.992087    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:43.004205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:43.004182    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:43.299462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:13:43.299359    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:13:50.805374 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:13:50.805337    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:14:01.711246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.711208    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"]
Oct 30 22:14:01.714672 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.714657    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.787894 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.787867    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-963c3a\"/\"openshift-service-ca.crt\""
Oct 30 22:14:01.809053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.809020    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.809215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.809086    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.809215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.809154    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7dwz\" (UniqueName: \"kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.809295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.809216    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.809295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.809244    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.809357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.809341    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.817461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.817439    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-963c3a\"/\"distinct-w3-963c3a-kserve-self-signed-certs\""
Oct 30 22:14:01.817461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.817439    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-963c3a\"/\"kube-root-ca.crt\""
Oct 30 22:14:01.817638 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.817440    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-963c3a\"/\"default-dockercfg-54nv5\""
Oct 30 22:14:01.820252 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.820228    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"]
Oct 30 22:14:01.910503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910470    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.910680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910554    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.910680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910607    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7dwz\" (UniqueName: \"kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.910680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910664    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.910850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910707    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.910850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910757    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.910950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.910856    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.911129 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.911111    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.911190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.911131    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.913019 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.912996    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.913372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.913351    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:01.978074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:01.978003    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7dwz\" (UniqueName: \"kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz\") pod \"distinct-w3-963c3a-kserve-79bd676699-gf5h6\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") " pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:02.027387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:02.027332    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:02.417064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:02.416878    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"]
Oct 30 22:14:02.418985 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:14:02.418958    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f1e0b03_63d1_4636_ab80_c8298ebe8a6f.slice/crio-b89ead380df8757e494bc0e37257d478287b94a74ee95e1513bffb0d0aff6635 WatchSource:0}: Error finding container b89ead380df8757e494bc0e37257d478287b94a74ee95e1513bffb0d0aff6635: Status 404 returned error can't find the container with id b89ead380df8757e494bc0e37257d478287b94a74ee95e1513bffb0d0aff6635
Oct 30 22:14:03.390467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:03.390434    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" event={"ID":"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f","Type":"ContainerStarted","Data":"b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15"}
Oct 30 22:14:03.390467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:03.390483    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" event={"ID":"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f","Type":"ContainerStarted","Data":"b89ead380df8757e494bc0e37257d478287b94a74ee95e1513bffb0d0aff6635"}
Oct 30 22:14:03.519068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:03.519013    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podStartSLOduration=2.518993928 podStartE2EDuration="2.518993928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:03.516973049 +0000 UTC m=+1174.504781313" watchObservedRunningTime="2026-10-30 22:14:03.518993928 +0000 UTC m=+1174.506802218"
Oct 30 22:14:04.028442 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:04.028379    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:12.028461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:12.028430    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:12.041851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:12.041820    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:12.444103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:12.444077    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:14:17.470108 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.470079    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"]
Oct 30 22:14:17.473731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.473712    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.519872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.519845    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"]
Oct 30 22:14:17.523271 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.523253    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.572279 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.572253    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"]
Oct 30 22:14:17.593180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593145    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.593327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593208    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593241    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593286    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.593533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593326    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhgfr\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593368    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxzts\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.593533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593427    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593490    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.593727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593535    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593599    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593651    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.593727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593678    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.593727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593710    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.594022 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593752    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.594022 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593791    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.594022 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.593846    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.594022 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.594015    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.594222 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.594114    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.626042 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.626007    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"]
Oct 30 22:14:17.701592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.701554    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.701592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.701597    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.701852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.701615    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.701852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.701650    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.701852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.701687    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhgfr\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.702136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702050    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.702136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702067    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.702136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702054    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.702136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702121    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxzts\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.702424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702160    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.702424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702247    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.702424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702359    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.702424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702417    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.702653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702430    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702825    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702879    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702901    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702929    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702970    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.702999    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.703041    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.703079    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.703101    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.703387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.703329    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.704065 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.704020    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.705799 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.704342    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.705799 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.704753    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.705799 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.704951    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.706577 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.706533    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.708845 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.708802    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.711196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.711168    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.724007 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.723955    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.765868 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.765843    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxzts\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.766000 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.765882    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhgfr\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.766064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.766041    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:17.767385 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.767366    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token\") pod \"e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") " pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.787167 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.787146    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:17.838211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:17.838091    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:18.139957 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.139872    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"]
Oct 30 22:14:18.144649 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.144631    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.207796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.207765    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"]
Oct 30 22:14:18.209698 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209670    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.209861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209715    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.209861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209760    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.209861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209778    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.209861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209843    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.209861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209859    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.210149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209883    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.210149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209927    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.210149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.209962    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsz86\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.223897 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.223872    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"]
Oct 30 22:14:18.224827 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:14:18.224804    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2064d62c_1633_47c4_b455_354994115829.slice/crio-021e72b9c2a72980c74b8f363b994284660044a2f3ef8cbfb48e2def34fe6eb0 WatchSource:0}: Error finding container 021e72b9c2a72980c74b8f363b994284660044a2f3ef8cbfb48e2def34fe6eb0: Status 404 returned error can't find the container with id 021e72b9c2a72980c74b8f363b994284660044a2f3ef8cbfb48e2def34fe6eb0
Oct 30 22:14:18.227389 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.227354    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.227507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.227454    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.227572 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.227503    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.280738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.280709    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"]
Oct 30 22:14:18.281338 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:14:18.281317    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8a888c_12d6_4b7a_8a8b_c63cb09b2341.slice/crio-ff948e323a27eed9e61e4c91d58419d1ac825d2429bf9c7481940f75f6fc6221 WatchSource:0}: Error finding container ff948e323a27eed9e61e4c91d58419d1ac825d2429bf9c7481940f75f6fc6221: Status 404 returned error can't find the container with id ff948e323a27eed9e61e4c91d58419d1ac825d2429bf9c7481940f75f6fc6221
Oct 30 22:14:18.284144 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.284106    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.284230 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.284189    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.284270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.284235    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.310767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.310736    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.310906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.310781    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.310906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.310821    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.310906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.310880    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.310925    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsz86\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.310974    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.311022    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311232 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.311078    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311232 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.311110    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311232 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.311143    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.311451    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.311942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.311850    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.312190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.312166    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.312268 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.312164    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.313835 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.313806    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.313960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.313941    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.380292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.380230    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsz86\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.381755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.381701    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.459149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.459123    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:18.484337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.483318    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" event={"ID":"2064d62c-1633-47c4-b455-354994115829","Type":"ContainerStarted","Data":"8ce213841e3d182c4c21e0e9aed68ef61798fdfe43d41f695554c2a48084d403"}
Oct 30 22:14:18.484337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.483380    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" event={"ID":"2064d62c-1633-47c4-b455-354994115829","Type":"ContainerStarted","Data":"021e72b9c2a72980c74b8f363b994284660044a2f3ef8cbfb48e2def34fe6eb0"}
Oct 30 22:14:18.500457 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.500390    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" event={"ID":"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341","Type":"ContainerStarted","Data":"0a99d6abc8e07e4a49f187fcb905d9e2a8f4ec91279ab8237942a64aae7d5090"}
Oct 30 22:14:18.500663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.500495    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" event={"ID":"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341","Type":"ContainerStarted","Data":"ff948e323a27eed9e61e4c91d58419d1ac825d2429bf9c7481940f75f6fc6221"}
Oct 30 22:14:18.614450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.613880    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podStartSLOduration=1.613858206 podStartE2EDuration="1.613858206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:18.611823527 +0000 UTC m=+1189.599631789" watchObservedRunningTime="2026-10-30 22:14:18.613858206 +0000 UTC m=+1189.601666469"
Oct 30 22:14:18.787353 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.787276    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:18.838617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.838584    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:18.894179 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.894119    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podStartSLOduration=1.8940964949999999 podStartE2EDuration="1.894096495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:18.74868047 +0000 UTC m=+1189.736488712" watchObservedRunningTime="2026-10-30 22:14:18.894096495 +0000 UTC m=+1189.881904760"
Oct 30 22:14:18.895576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.895540    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"]
Oct 30 22:14:18.898295 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:14:18.898164    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b86d04_856a_484a_8cb4_f9092faf55d7.slice/crio-5b3e007e825d65b5b199453409936b36e8167ea87bf600bb346fddf995b22c54 WatchSource:0}: Error finding container 5b3e007e825d65b5b199453409936b36e8167ea87bf600bb346fddf995b22c54: Status 404 returned error can't find the container with id 5b3e007e825d65b5b199453409936b36e8167ea87bf600bb346fddf995b22c54
Oct 30 22:14:18.901208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.901166    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.901322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.901238    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:18.901322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:18.901275    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:19.536423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.535829    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" event={"ID":"13b86d04-856a-484a-8cb4-f9092faf55d7","Type":"ContainerStarted","Data":"56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe"}
Oct 30 22:14:19.536916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.536687    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" event={"ID":"13b86d04-856a-484a-8cb4-f9092faf55d7","Type":"ContainerStarted","Data":"5b3e007e825d65b5b199453409936b36e8167ea87bf600bb346fddf995b22c54"}
Oct 30 22:14:19.697421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.697013    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podStartSLOduration=1.69699294 podStartE2EDuration="1.69699294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:19.695704235 +0000 UTC m=+1190.683512524" watchObservedRunningTime="2026-10-30 22:14:19.69699294 +0000 UTC m=+1190.684801203"
Oct 30 22:14:19.807507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.807173    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 22:14:19.807507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.807248    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 22:14:19.840053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.840017    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:14:19.840240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:19.840081    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:14:20.461243 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.461208    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:20.461554 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.461524    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:20.461655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.461599    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:20.790915 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.790823    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 22:14:20.791530 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.791498    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 22:14:20.841426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.838901    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:14:20.841426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:20.838974    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:14:20.869514 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:14:20.869483    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:14:21.460064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:21.460035    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:21.460280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:21.460093    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:21.793479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:21.790815    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 22:14:21.793479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:21.790893    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 22:14:21.839422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:21.838803    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:14:21.839422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:21.838862    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:14:22.459757 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:22.459719    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:22.459939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:22.459778    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:22.792020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:22.791929    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 22:14:22.792276 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:22.792247    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 22:14:22.839907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:22.839784    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:14:22.839907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:22.839845    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:14:23.473422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:23.460837    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:23.473422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:23.460902    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:23.791474 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:23.788943    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 22:14:23.791474 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:23.789024    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 22:14:23.844423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:23.844213    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:14:23.844423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:23.844275    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:14:24.468364 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:24.468331    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:24.468871 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:24.468842    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:24.787893 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:24.787801    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 22:14:24.788129 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:24.788102    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 22:14:24.839308 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:24.839272    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 22:14:24.839528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:24.839344    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 22:14:25.460091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:25.460055    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:25.460528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:25.460118    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:26.039880 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.039845    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:26.042295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.042275    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:26.043204 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.043185    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:14:26.063375 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.063356    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:26.064143 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.064123    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:26.065159 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.065143    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:14:26.459899 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.459865    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 22:14:26.460068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:26.459946    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 22:14:27.464121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:27.464090    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:27.464562 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:27.464500    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:27.465239 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:27.465223    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:14:49.018152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.018077    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"]
Oct 30 22:14:49.021867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.021847    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075265 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075236    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075284    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075353    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075388    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075452    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075498    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075532    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075576    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqv8\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.075745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.075650    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.124603 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.124571    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-963c3a-openshift-default-dockercfg-fd9jr\""
Oct 30 22:14:49.127545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.127511    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"]
Oct 30 22:14:49.176871 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.176843    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.176888    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.176920    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqv8\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.176966    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177008    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177038    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177074    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177107    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177134    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177250    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177301    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177625 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177473    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177625 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177561    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.177758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.177739    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.179934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.179905    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.180234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.180215    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.239491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.239464    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.239647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.239586    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqv8\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.334456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.334370    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:49.796898 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.796876    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"]
Oct 30 22:14:49.798742 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:14:49.798714    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode340d27b_4200_4797_b166_d003fc6ada89.slice/crio-69c88028fb127bb8af826cbfa0057cd1484ff81dfe2730a163806e7cd03c3def WatchSource:0}: Error finding container 69c88028fb127bb8af826cbfa0057cd1484ff81dfe2730a163806e7cd03c3def: Status 404 returned error can't find the container with id 69c88028fb127bb8af826cbfa0057cd1484ff81dfe2730a163806e7cd03c3def
Oct 30 22:14:49.801067 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.801037    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:49.801129 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.801100    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:49.801166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:49.801149    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:14:50.799370 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:50.799328    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" event={"ID":"e340d27b-4200-4797-b166-d003fc6ada89","Type":"ContainerStarted","Data":"f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db"}
Oct 30 22:14:50.799869 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:50.799388    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" event={"ID":"e340d27b-4200-4797-b166-d003fc6ada89","Type":"ContainerStarted","Data":"69c88028fb127bb8af826cbfa0057cd1484ff81dfe2730a163806e7cd03c3def"}
Oct 30 22:14:50.819505 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:14:50.819476    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:14:51.335449 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:51.335374    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:51.337132 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:51.337106    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:14:51.337265 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:51.337154    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:14:52.335241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:52.335197    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:14:52.335693 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:52.335262    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:14:53.335567 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:53.335532    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:14:53.335962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:53.335587    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:14:54.335187 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:54.335155    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:14:54.335389 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:54.335212    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:14:55.334890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:55.334852    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:14:55.335286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:55.334909    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:14:56.335323 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:56.335282    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 22:14:56.335768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:56.335345    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 22:14:57.340097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:57.340069    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:57.340596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:57.340563    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:57.341470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:57.341450    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:14:57.520910 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:57.520852    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podStartSLOduration=9.520838152 podStartE2EDuration="9.520838152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:50.987797368 +0000 UTC m=+1221.975605720" watchObservedRunningTime="2026-10-30 22:14:57.520838152 +0000 UTC m=+1228.508646414"
Oct 30 22:14:59.277958 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.277926    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"]
Oct 30 22:14:59.282731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.282714    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.337196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.337160    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-77cb1f-openshift-default-dockercfg-ttk97\""
Oct 30 22:14:59.389858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.389824    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"]
Oct 30 22:14:59.404361 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404317    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404549 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404382    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404549 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404511    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4phl\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404577    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404628    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404711    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404770 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404742    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404803 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404776    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.404960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.404941    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506110    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506145    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506385 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506167    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506385 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506271    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506385 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506337    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506425    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506478    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4phl\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506516    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506566    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506605    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.506762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506642    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.507017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.506992    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.507123 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.507077    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.507345 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.507324    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.508906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.508888    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.509052 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.509037    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.577460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.576969    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.579970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.579890    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4phl\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:14:59.597683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:14:59.597646    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:15:00.040561 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:00.040529    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"]
Oct 30 22:15:00.042548 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:15:00.042515    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c9c649_98e4_41fc_a3bc_9adab9962a3f.slice/crio-fdd6861c9412c52f099154d2c7442bce3906bd5490089ff4ab0ba78bb9f96530 WatchSource:0}: Error finding container fdd6861c9412c52f099154d2c7442bce3906bd5490089ff4ab0ba78bb9f96530: Status 404 returned error can't find the container with id fdd6861c9412c52f099154d2c7442bce3906bd5490089ff4ab0ba78bb9f96530
Oct 30 22:15:00.044843 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:00.044812    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:15:00.044915 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:00.044878    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:15:00.044953 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:00.044915    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:15:00.951448 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:00.940126    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" event={"ID":"f7c9c649-98e4-41fc-a3bc-9adab9962a3f","Type":"ContainerStarted","Data":"c96b9d4cd0ccee819f04fbd3ce55463e55ffb9f3f9658d11e13d34ff51c93ef4"}
Oct 30 22:15:00.951977 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:00.951939    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" event={"ID":"f7c9c649-98e4-41fc-a3bc-9adab9962a3f","Type":"ContainerStarted","Data":"fdd6861c9412c52f099154d2c7442bce3906bd5490089ff4ab0ba78bb9f96530"}
Oct 30 22:15:01.105877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:01.105805    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podStartSLOduration=2.105781296 podStartE2EDuration="2.105781296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.101730109 +0000 UTC m=+1232.089538373" watchObservedRunningTime="2026-10-30 22:15:01.105781296 +0000 UTC m=+1232.093589560"
Oct 30 22:15:01.599643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:01.599601    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:15:01.600842 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:01.600817    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:15:01.600981 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:01.600891    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:15:02.598226 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:02.598191    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:15:02.598875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:02.598844    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:15:03.598470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:03.598412    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:15:03.598973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:03.598495    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:15:04.598524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:04.598494    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:15:04.598924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:04.598547    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:15:05.601425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:05.598843    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:15:05.601425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:05.598908    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:15:06.598216 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:06.598182    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 22:15:06.598414 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:06.598235    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 22:15:07.603147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:07.603118    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:15:07.603618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:07.603422    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:15:07.604281 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:07.604262    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:15:20.805834 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:15:20.805793    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:15:32.700553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.700501    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"]
Oct 30 22:15:32.706877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.706851    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.760939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.760902    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2a4123-openshift-default-dockercfg-ldqs2\""
Oct 30 22:15:32.812551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.812510    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"]
Oct 30 22:15:32.847711 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.847679    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.847924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.847718    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.847924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.847815    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.848048 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.847938    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.848048 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.848030    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.848228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.848062    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.848228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.848129    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.848228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.848174    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.848228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.848200    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd7lt\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950086 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950049    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950112    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950174    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950196    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd7lt\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950315    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950348    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950414    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950459    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950540    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950600    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950824    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950886    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.950954 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.950904    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.951037 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.951016    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.952858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.952837    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:32.953359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:32.953326    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:33.023232 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.023192    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:33.023426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.023261    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd7lt\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:33.320284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.320193    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:33.753121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.753080    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"]
Oct 30 22:15:33.754023 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:15:33.753992    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744e95b0_6cc9_4160_898e_a7aab48c54d3.slice/crio-42aa0f475ebfbe483d069d753ce6ea87fd8cdfe1f69d9aed82eb56ff3f8db61c WatchSource:0}: Error finding container 42aa0f475ebfbe483d069d753ce6ea87fd8cdfe1f69d9aed82eb56ff3f8db61c: Status 404 returned error can't find the container with id 42aa0f475ebfbe483d069d753ce6ea87fd8cdfe1f69d9aed82eb56ff3f8db61c
Oct 30 22:15:33.756521 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.756488    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:15:33.756604 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.756565    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:15:33.756649 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:33.756601    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:15:34.120449 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:34.120328    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" event={"ID":"744e95b0-6cc9-4160-898e-a7aab48c54d3","Type":"ContainerStarted","Data":"12174d3961463f4f1947241d4e386035dbad627ff1d31512403c22670bd3a705"}
Oct 30 22:15:34.120449 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:34.120380    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" event={"ID":"744e95b0-6cc9-4160-898e-a7aab48c54d3","Type":"ContainerStarted","Data":"42aa0f475ebfbe483d069d753ce6ea87fd8cdfe1f69d9aed82eb56ff3f8db61c"}
Oct 30 22:15:34.254573 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:34.254511    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podStartSLOduration=2.254493048 podStartE2EDuration="2.254493048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:34.251920481 +0000 UTC m=+1265.239728748" watchObservedRunningTime="2026-10-30 22:15:34.254493048 +0000 UTC m=+1265.242301311"
Oct 30 22:15:34.320617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:34.320585    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:34.322208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:34.322184    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:34.322320 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:34.322235    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:35.321384 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:35.321351    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:35.321900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:35.321442    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:36.320619 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:36.320586    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:36.320800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:36.320646    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:37.321456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:37.321381    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:37.321870 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:37.321480    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:38.320816 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:38.320779    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:38.321106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:38.321075    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:39.321381 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:39.321340    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:39.321829 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:39.321448    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:40.321791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:40.321752    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 22:15:40.322227 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:40.321824    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 22:15:41.324851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:41.324820    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:41.325260 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:41.325196    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:41.325897 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:15:41.325879    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:15:50.805247 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:15:50.805209    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:16:05.125180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.125137    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"]
Oct 30 22:16:05.132181 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.132143    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.186248 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.186216    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"]
Oct 30 22:16:05.233121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.233089    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c76d9e\"/\"distinct-2-w6-c76d9e-kserve-self-signed-certs\""
Oct 30 22:16:05.233300 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.233133    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c76d9e\"/\"default-dockercfg-s8b25\""
Oct 30 22:16:05.234749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.234735    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c76d9e\"/\"openshift-service-ca.crt\""
Oct 30 22:16:05.234809 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.234747    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c76d9e\"/\"kube-root-ca.crt\""
Oct 30 22:16:05.243349 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.243310    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.243492 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.243360    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.243535 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.243492    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.243535 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.243523    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.243627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.243551    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9d9p\" (UniqueName: \"kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.243673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.243634    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.344980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.344946    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.344997    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345025    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345041    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345067    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9d9p\" (UniqueName: \"kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345097    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345546 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345520    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345612 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345548    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.345612 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.345571    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.347549 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.347521    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.347775 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.347758    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.408048 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.407977    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9d9p\" (UniqueName: \"kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p\") pod \"distinct-2-w6-c76d9e-kserve-855f476566-7q9lx\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") " pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.447508 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.447476    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:05.931867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:05.931845    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"]
Oct 30 22:16:05.933552 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:16:05.933524    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1f76beb_38b7_4318_b025_5669ffaff916.slice/crio-b2f475ff318634c4ced45754ca0b15f6ae28796fbd2fd1859d254e86a382a85d WatchSource:0}: Error finding container b2f475ff318634c4ced45754ca0b15f6ae28796fbd2fd1859d254e86a382a85d: Status 404 returned error can't find the container with id b2f475ff318634c4ced45754ca0b15f6ae28796fbd2fd1859d254e86a382a85d
Oct 30 22:16:06.365277 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:06.365240    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" event={"ID":"a1f76beb-38b7-4318-b025-5669ffaff916","Type":"ContainerStarted","Data":"6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e"}
Oct 30 22:16:06.365709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:06.365283    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" event={"ID":"a1f76beb-38b7-4318-b025-5669ffaff916","Type":"ContainerStarted","Data":"b2f475ff318634c4ced45754ca0b15f6ae28796fbd2fd1859d254e86a382a85d"}
Oct 30 22:16:06.447807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:06.447779    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:06.516042 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:06.515983    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podStartSLOduration=1.5159630819999999 podStartE2EDuration="1.515963082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:06.514025305 +0000 UTC m=+1297.501833569" watchObservedRunningTime="2026-10-30 22:16:06.515963082 +0000 UTC m=+1297.503771346"
Oct 30 22:16:15.448092 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:15.448010    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:15.460329 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:15.460306    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:16.430582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:16.430557    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:16:19.120069 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.120034    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"]
Oct 30 22:16:19.123874 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.123855    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.231507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.231468    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"]
Oct 30 22:16:19.236422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.236378    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.236590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.236558    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.236727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.236695    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.236792 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.236751    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.236900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.236885    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.236962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.236925    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxwh\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.237082 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.237060    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.237142 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.237125    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.237325 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.237298    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338071    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338121    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338178    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338212    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxwh\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338263    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338519 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338300    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338519 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338434    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338519 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338484    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338543    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338655    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338726    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338926 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338888    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.338967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.338929    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.339336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.339314    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.340636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.340608    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.340906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.340891    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.401133 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.401063    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.401326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.401296    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxwh\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh\") pod \"e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") " pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.437364 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.437341    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:19.870935 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.870904    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"]
Oct 30 22:16:19.871635 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:16:19.871609    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac46abd_764c_465d_9952_af513f3af401.slice/crio-6c7565f800cf78ef881d4dc023f9f96ddf2d787100f81eebdca971dfebd831fb WatchSource:0}: Error finding container 6c7565f800cf78ef881d4dc023f9f96ddf2d787100f81eebdca971dfebd831fb: Status 404 returned error can't find the container with id 6c7565f800cf78ef881d4dc023f9f96ddf2d787100f81eebdca971dfebd831fb
Oct 30 22:16:19.874020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.873983    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:16:19.874119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.874066    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:16:19.874119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:19.874113    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:16:20.446075 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:20.446044    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" event={"ID":"9ac46abd-764c-465d-9952-af513f3af401","Type":"ContainerStarted","Data":"bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7"}
Oct 30 22:16:20.446563 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:20.446094    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" event={"ID":"9ac46abd-764c-465d-9952-af513f3af401","Type":"ContainerStarted","Data":"6c7565f800cf78ef881d4dc023f9f96ddf2d787100f81eebdca971dfebd831fb"}
Oct 30 22:16:20.669597 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:20.669546    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podStartSLOduration=1.669532905 podStartE2EDuration="1.669532905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:20.619002604 +0000 UTC m=+1311.606810877" watchObservedRunningTime="2026-10-30 22:16:20.669532905 +0000 UTC m=+1311.657341168"
Oct 30 22:16:20.821203 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:16:20.821110    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:16:21.438381 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:21.438353    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:21.440006 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:21.439978    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 22:16:21.440142 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:21.440044    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 22:16:22.438505 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:22.438466    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 22:16:22.438907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:22.438532    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 22:16:23.437991 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:23.437955    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 22:16:23.438177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:23.438027    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 22:16:24.438639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:24.438607    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 22:16:24.439054 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:24.438658    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 22:16:25.438258 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:25.438220    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 22:16:25.438447 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:25.438282    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 22:16:26.438350 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:26.438312    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 22:16:26.438846 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:26.438387    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 22:16:27.442490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:27.442461    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:27.442970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:27.442951    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:27.443881 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:27.443762    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:16:34.289422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:34.289281    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 30 22:16:34.463445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:34.463154    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:34.483421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:34.482520    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:34.492293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:34.491110    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:34.516543 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:34.516480    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:16:45.218723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.218689    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"]
Oct 30 22:16:45.224175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.224147    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.329988 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.329945    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"]
Oct 30 22:16:45.364228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364192    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364272    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364352    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364385    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364499    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364544    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364600    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364650    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr8ww\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.364761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.364669    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466085    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466130    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466149    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466182    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466208    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466232    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466288    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr8ww\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466671 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466316    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466671 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466417    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466759    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466790    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.466875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.466862    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.467374 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.467347    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.467547 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.467528    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.468878 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.468823    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.468995 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.468972    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.529472 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.529443    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.529946 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.529927    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr8ww\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.537861 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.537839    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:45.973037 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:16:45.973009    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ecab24_c840_4db1_9253_f3c9aa15ab0c.slice/crio-5991fda51d3d4964ab124f31e7ad33744289ca1d0faee4e5a237ebde7f32f61f WatchSource:0}: Error finding container 5991fda51d3d4964ab124f31e7ad33744289ca1d0faee4e5a237ebde7f32f61f: Status 404 returned error can't find the container with id 5991fda51d3d4964ab124f31e7ad33744289ca1d0faee4e5a237ebde7f32f61f
Oct 30 22:16:45.973147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.973110    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"]
Oct 30 22:16:45.975451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.975380    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:16:45.975541 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.975489    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:16:45.975541 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:45.975518    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:16:46.637683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:46.637647    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" event={"ID":"f5ecab24-c840-4db1-9253-f3c9aa15ab0c","Type":"ContainerStarted","Data":"4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb"}
Oct 30 22:16:46.638165 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:46.637702    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" event={"ID":"f5ecab24-c840-4db1-9253-f3c9aa15ab0c","Type":"ContainerStarted","Data":"5991fda51d3d4964ab124f31e7ad33744289ca1d0faee4e5a237ebde7f32f61f"}
Oct 30 22:16:46.814615 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:46.814565    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podStartSLOduration=1.814550458 podStartE2EDuration="1.814550458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:46.813351213 +0000 UTC m=+1337.801159494" watchObservedRunningTime="2026-10-30 22:16:46.814550458 +0000 UTC m=+1337.802358721"
Oct 30 22:16:47.538858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:47.538830    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:47.540559 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:47.540527    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:16:47.540688 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:47.540581    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:16:48.538504 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:48.538472    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:16:48.538900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:48.538537    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:16:49.538984 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:49.538952    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:16:49.539370 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:49.539015    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:16:50.538997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:50.538966    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:16:50.539409 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:50.539033    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:16:50.821225 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:16:50.821124    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:16:51.223270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:51.223240    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"]
Oct 30 22:16:51.223471 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:51.223447    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy" containerID="cri-o://8f9262ddfddfb8b8cf6a78eebfaef9a626d2f0f58a41c29e3d7026c2db6ec73b" gracePeriod=30
Oct 30 22:16:51.539254 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:51.539175    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:16:51.539660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:51.539255    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:16:52.538749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:52.538715    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 22:16:52.538952 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:52.538776    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 22:16:53.543020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:53.542990    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:53.543505 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:53.543417    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:53.544111 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:53.544094    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:16:56.814592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:56.812581    4536 generic.go:356] "Generic (PLEG): container finished" podID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerID="8f9262ddfddfb8b8cf6a78eebfaef9a626d2f0f58a41c29e3d7026c2db6ec73b" exitCode=0
Oct 30 22:16:56.814592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:56.812767    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" event={"ID":"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282","Type":"ContainerDied","Data":"8f9262ddfddfb8b8cf6a78eebfaef9a626d2f0f58a41c29e3d7026c2db6ec73b"}
Oct 30 22:16:57.347608 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.347575    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.384792    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.384856    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.384915    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.385041    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.385094    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.385159    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.385278    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.385325    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.385356    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo\") pod \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\" (UID: \"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282\") "
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.390690    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:57.393366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.390904    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:57.412699 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.408793    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:57.412699 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.409051    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:57.421820 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.421788    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:57.452283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.443944    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:57.452283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.447843    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:57.452283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.452150    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "kube-api-access-9qvq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:57.469056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.467514    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy" pod "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" (UID: "05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500333    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500376    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500410    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500425    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qvq2\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-kube-api-access-9qvq2\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500439    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500451    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500463    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500476    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.500576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.500489    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.925944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.885951    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55" event={"ID":"05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282","Type":"ContainerDied","Data":"adf80efed1926bbdf61a8951c8bd8d3a563599ed60c65417a4ac8dd6398c50ae"}
Oct 30 22:16:57.925944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.886024    4536 scope.go:122] "RemoveContainer" containerID="8f9262ddfddfb8b8cf6a78eebfaef9a626d2f0f58a41c29e3d7026c2db6ec73b"
Oct 30 22:16:57.925944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:57.886222    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"
Oct 30 22:16:58.086259 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:58.086221    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"]
Oct 30 22:16:58.134135 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:58.134100    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-28k55"]
Oct 30 22:16:59.174270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:16:59.174239    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" path="/var/lib/kubelet/pods/05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282/volumes"
Oct 30 22:17:05.176695 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.176651    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"]
Oct 30 22:17:05.177180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.177167    4536 cpu_manager.go:414] "containerMap: removing container" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy"
Oct 30 22:17:05.177237 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.177181    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy"
Oct 30 22:17:05.177355 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.177345    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="05ff9e68-3c3e-46c2-ac8d-21fd5f4c4282" containerName="istio-proxy"
Oct 30 22:17:05.182176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.182147    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256157 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256114    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256356 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256191    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256356 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256253    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256356 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256291    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256356 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256320    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256835 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256809    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkxct\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256928    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.256969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.256957    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.257067 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.257018    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.290902 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.290857    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"]
Oct 30 22:17:05.358062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358026    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358072    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358103    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358137    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358157    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358226    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkxct\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358272    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358675 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358294    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358675 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358320    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358675 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358567    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358823 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358673    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358823 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358808    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358953 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358881    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.358953 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.358924    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.360712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.360685    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.360934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.360915    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.420615 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.420587    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.420771 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.420740    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkxct\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.501833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.501796    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:05.952210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.952177    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"]
Oct 30 22:17:05.953999 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:05.953973    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d209b30_5473_4ee4_aa77_395fcaf10225.slice/crio-9f68f45c4ce9f5be8761ee116020de37e48efc41ab9cf7ced5066f440acf1b4f WatchSource:0}: Error finding container 9f68f45c4ce9f5be8761ee116020de37e48efc41ab9cf7ced5066f440acf1b4f: Status 404 returned error can't find the container with id 9f68f45c4ce9f5be8761ee116020de37e48efc41ab9cf7ced5066f440acf1b4f
Oct 30 22:17:05.956616 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.956581    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:05.956719 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.956689    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:05.956778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:05.956737    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:06.939722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:06.939684    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" event={"ID":"7d209b30-5473-4ee4-aa77-395fcaf10225","Type":"ContainerStarted","Data":"31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d"}
Oct 30 22:17:06.940227 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:06.939757    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" event={"ID":"7d209b30-5473-4ee4-aa77-395fcaf10225","Type":"ContainerStarted","Data":"9f68f45c4ce9f5be8761ee116020de37e48efc41ab9cf7ced5066f440acf1b4f"}
Oct 30 22:17:07.115357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:07.115291    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podStartSLOduration=2.1152707570000002 podStartE2EDuration="2.115270757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:07.112699858 +0000 UTC m=+1358.100508121" watchObservedRunningTime="2026-10-30 22:17:07.115270757 +0000 UTC m=+1358.103079022"
Oct 30 22:17:07.502782 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:07.502742    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:07.504423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:07.504380    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:17:07.504593 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:07.504451    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:17:08.502553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:08.502522    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:17:08.503090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:08.502575    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:17:09.502479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:09.502438    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:17:09.502682 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:09.502490    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:17:10.502858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:10.502822    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:17:10.503245 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:10.502876    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:17:11.502284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:11.502250    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:17:11.502496 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:11.502324    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:17:12.502859 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:12.502815    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 22:17:12.503304 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:12.502896    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 22:17:13.507758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:13.507729    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:13.508190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:13.508130    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:13.508976 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:13.508952    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:17:15.485903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.485873    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"]
Oct 30 22:17:15.490102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.490085    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.617190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.610995    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wscn\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.617530 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.617495    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.617783 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.617754    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.617901 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.617799    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.617901 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.617831    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.618014 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.617978    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.618116 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.618096    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.618241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.618161    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.618241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.618191    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.619188 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.619167    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"]
Oct 30 22:17:15.719481 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719454    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wscn\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719497    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719541    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719563    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719589    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719624    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719952 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719721    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.719952 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.719773    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.720083 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.720001    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.720083 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.720017    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.720198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.720083    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.720198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.720163    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.720315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.720205    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.720495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.720476    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.722136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.722116    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.722566 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.722548    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.785148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.785078    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.785879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.785859    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wscn\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn\") pod \"e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") " pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:15.803908 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:15.803884    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:16.286063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:16.286038    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"]
Oct 30 22:17:16.287335 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:16.287307    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86de333_9735_452c_aa27_c15903aa2717.slice/crio-35fbb0c90c5539235dafb4913ea7314d6dab88e84122fffbeba3a460866aa8ae WatchSource:0}: Error finding container 35fbb0c90c5539235dafb4913ea7314d6dab88e84122fffbeba3a460866aa8ae: Status 404 returned error can't find the container with id 35fbb0c90c5539235dafb4913ea7314d6dab88e84122fffbeba3a460866aa8ae
Oct 30 22:17:16.290487 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:16.290447    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:16.290564 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:16.290522    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:16.290564 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:16.290554    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:17.006587 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:17.006554    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" event={"ID":"a86de333-9735-452c-aa27-c15903aa2717","Type":"ContainerStarted","Data":"8201347c6a9a5a5f5a0f415c13bc311eb586b80ff87163fdde0b93cb74583934"}
Oct 30 22:17:17.006587 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:17.006604    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" event={"ID":"a86de333-9735-452c-aa27-c15903aa2717","Type":"ContainerStarted","Data":"35fbb0c90c5539235dafb4913ea7314d6dab88e84122fffbeba3a460866aa8ae"}
Oct 30 22:17:17.258347 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:17.258241    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podStartSLOduration=2.258221737 podStartE2EDuration="2.258221737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:17.20610008 +0000 UTC m=+1368.193908344" watchObservedRunningTime="2026-10-30 22:17:17.258221737 +0000 UTC m=+1368.246029999"
Oct 30 22:17:17.804730 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:17.804699    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:17.806252 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:17.806228    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:17:17.807451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:17.806771    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:17:18.632304 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.632273    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"]
Oct 30 22:17:18.637302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.637275    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.752716 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.752678    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"]
Oct 30 22:17:18.763004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.762970    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763036    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763155    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763227    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4xhn\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763460    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763564 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763547    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763604 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763587    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763638    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.763758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.763740    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.804985 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.804945    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:17:18.805166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.804997    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:17:18.864768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.864738    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.864768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.864776    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.864818    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.864838    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.864855    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4xhn\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865001    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865101    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865146    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865136    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865267    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865587 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865552    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865940 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865673    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865940 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865818    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.865940 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.865832    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.867558 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.867475    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.867911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.867889    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.927996 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.927918    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.928160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.928096    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4xhn\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn\") pod \"e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") " pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:18.952002 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:18.951977    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:19.415527 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:19.415464    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"]
Oct 30 22:17:19.416266 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:19.416237    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d307b9c_aae8_46ed_b511_8094d566bd45.slice/crio-92b4898ae0238229761dde6322a37522eb7d5c382c092cae237668dc042aab01 WatchSource:0}: Error finding container 92b4898ae0238229761dde6322a37522eb7d5c382c092cae237668dc042aab01: Status 404 returned error can't find the container with id 92b4898ae0238229761dde6322a37522eb7d5c382c092cae237668dc042aab01
Oct 30 22:17:19.419165 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:19.419131    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:19.419263 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:19.419226    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:19.419308 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:19.419269    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:19.808423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:19.807759    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:17:19.808423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:19.807818    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:17:20.031811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.031772    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" event={"ID":"1d307b9c-aae8-46ed-b511-8094d566bd45","Type":"ContainerStarted","Data":"13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f"}
Oct 30 22:17:20.032009 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.031834    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" event={"ID":"1d307b9c-aae8-46ed-b511-8094d566bd45","Type":"ContainerStarted","Data":"92b4898ae0238229761dde6322a37522eb7d5c382c092cae237668dc042aab01"}
Oct 30 22:17:20.161377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.161285    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podStartSLOduration=2.161269929 podStartE2EDuration="2.161269929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:20.158012958 +0000 UTC m=+1371.145821222" watchObservedRunningTime="2026-10-30 22:17:20.161269929 +0000 UTC m=+1371.149078241"
Oct 30 22:17:20.806422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.805807    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:17:20.806422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.805869    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:17:20.849694 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:20.849658    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:17:20.952407 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.952363    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:20.953611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.953592    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:17:20.953689 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:20.953636    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:17:21.807421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:21.804906    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:17:21.807421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:21.804979    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:17:21.953334 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:21.953305    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:17:21.953841 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:21.953810    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:17:22.804436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:22.804347    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 22:17:22.804436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:22.804419    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 22:17:22.952930 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:22.952894    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:17:22.953103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:22.952958    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:17:23.814679 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:23.814650    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:23.815202 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:23.815058    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:23.816137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:23.816116    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:17:23.952322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:23.952285    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:17:23.952510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:23.952354    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:17:24.952672 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:24.952642    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:17:24.953057 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:24.952694    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:17:25.952464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:25.952429    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:17:25.952709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:25.952498    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:17:26.969043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:26.969009    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:26.969866 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:26.969829    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:26.970722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:26.970698    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:17:32.288573 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.288500    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.291545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.291165    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Oct 30 22:17:32.424636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.424569    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.441551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.441493    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.457614 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.457545    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.615627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.615515    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.700796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.700750    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.913676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.913099    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:32.943676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:32.943604    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:33.034634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.034534    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:33.294548 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.294499    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:33.296429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.295640    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:17:33.296689 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.295148    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 30 22:17:33.297518 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.297500    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:17:33.426862 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.426482    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 30 22:17:33.427062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.426915    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:17:33.444565 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.444515    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:17:33.446217 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.445184    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:17:33.505424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.488658    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:33.505424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.489046    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:17:33.959640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.959596    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:33.968423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.960444    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:17:34.047420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.993889    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.087882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.996463    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Oct 30 22:17:34.088182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:33.996502    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": context deadline exceeded"
Oct 30 22:17:34.114590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.113594    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:17:34.350488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.311968    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.397452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.344497    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.451457 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.434370    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.457852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.453067    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.498438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.490481    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.758233 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.758184    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:34.983460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:34.976174    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:35.119381 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.119270    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:35.397607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.397506    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:35.536962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.429583    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:35.567046 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.566829    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:35.674065 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.673985    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:17:35.676469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.676448    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:17:35.677500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.677484    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:17:35.871314 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:35.871283    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:17:36.036972 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:36.036946    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:17:36.109342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:36.109314    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:17:36.112702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:36.112684    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:17:41.283251 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.283178    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-809de1"]
Oct 30 22:17:41.287440 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.287422    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:41.341931 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.341899    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tmcpb\""
Oct 30 22:17:41.343921 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.343897    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-809de1"]
Oct 30 22:17:41.405094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.405062    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbmwm\" (UniqueName: \"kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm\") pod \"test-curl-7426-809de1\" (UID: \"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0\") " pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:41.507142 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.507110    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbmwm\" (UniqueName: \"kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm\") pod \"test-curl-7426-809de1\" (UID: \"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0\") " pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:41.573644 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.573563    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbmwm\" (UniqueName: \"kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm\") pod \"test-curl-7426-809de1\" (UID: \"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0\") " pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:41.599724 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.599698    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:41.974617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.974598    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-809de1"]
Oct 30 22:17:41.976677 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:41.976653    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ce58bb5_442d_4dc3_b28e_6e7ef9efead0.slice/crio-2c27cac122e206d4a134fd60a87d1396fac4164f3937f47a0e3088430aec74d9 WatchSource:0}: Error finding container 2c27cac122e206d4a134fd60a87d1396fac4164f3937f47a0e3088430aec74d9: Status 404 returned error can't find the container with id 2c27cac122e206d4a134fd60a87d1396fac4164f3937f47a0e3088430aec74d9
Oct 30 22:17:41.978608 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:41.978593    4536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:17:42.650461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:42.650427    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-809de1" event={"ID":"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0","Type":"ContainerStarted","Data":"2c27cac122e206d4a134fd60a87d1396fac4164f3937f47a0e3088430aec74d9"}
Oct 30 22:17:45.673141 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:45.673102    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-809de1" event={"ID":"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0","Type":"ContainerStarted","Data":"97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387"}
Oct 30 22:17:45.789219 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:45.789165    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7426-809de1" podStartSLOduration=1.892174979 podStartE2EDuration="4.789151493s" totalImagesPullingTime="2.896976514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:45.78854364 +0000 UTC m=+1396.776351910" watchObservedRunningTime="2026-10-30 22:17:45.789151493 +0000 UTC m=+1396.776959756"
Oct 30 22:17:47.253433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.253390    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-809de1"]
Oct 30 22:17:47.260076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.260044    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-809de1"]
Oct 30 22:17:47.433200 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.433163    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-134d62"]
Oct 30 22:17:47.434129 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.434105    4536 cpu_manager.go:414] "containerMap: removing container" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" containerName="test-curl-7426-809de1"
Oct 30 22:17:47.434179 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.434135    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" containerName="test-curl-7426-809de1"
Oct 30 22:17:47.434314 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.434286    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" containerName="test-curl-7426-809de1"
Oct 30 22:17:47.438354 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.438337    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:47.494112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.494085    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-134d62"]
Oct 30 22:17:47.597694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.597619    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttncr\" (UniqueName: \"kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr\") pod \"test-curl-7426-134d62\" (UID: \"e64343e3-73c3-47d6-9bfc-c964ede522b4\") " pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:47.684488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.684431    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7426-809de1" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" containerName="test-curl-7426-809de1" containerID="cri-o://97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387" gracePeriod=2
Oct 30 22:17:47.699366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.699340    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttncr\" (UniqueName: \"kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr\") pod \"test-curl-7426-134d62\" (UID: \"e64343e3-73c3-47d6-9bfc-c964ede522b4\") " pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:47.735870 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:47.735829    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-809de1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:47.762163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:47.762131    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttncr\" (UniqueName: \"kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr\") pod \"test-curl-7426-134d62\" (UID: \"e64343e3-73c3-47d6-9bfc-c964ede522b4\") " pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:48.050495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:48.050465    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:48.427151 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:48.427116    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-134d62"]
Oct 30 22:17:48.428193 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:48.428166    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode64343e3_73c3_47d6_9bfc_c964ede522b4.slice/crio-8451c1321128e7c8f749cf7793090991386480bd344826de9d6c441644ba6718 WatchSource:0}: Error finding container 8451c1321128e7c8f749cf7793090991386480bd344826de9d6c441644ba6718: Status 404 returned error can't find the container with id 8451c1321128e7c8f749cf7793090991386480bd344826de9d6c441644ba6718
Oct 30 22:17:48.691485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:48.691385    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-134d62" event={"ID":"e64343e3-73c3-47d6-9bfc-c964ede522b4","Type":"ContainerStarted","Data":"399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b"}
Oct 30 22:17:48.691485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:48.691451    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-134d62" event={"ID":"e64343e3-73c3-47d6-9bfc-c964ede522b4","Type":"ContainerStarted","Data":"8451c1321128e7c8f749cf7793090991386480bd344826de9d6c441644ba6718"}
Oct 30 22:17:48.742552 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:48.742513    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-809de1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:48.858747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:48.858695    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7426-134d62" podStartSLOduration=1.858681107 podStartE2EDuration="1.858681107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:48.857002022 +0000 UTC m=+1399.844810282" watchObservedRunningTime="2026-10-30 22:17:48.858681107 +0000 UTC m=+1399.846489670"
Oct 30 22:17:49.212639 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:49.212603    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-809de1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:49.526475 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.523313    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"]
Oct 30 22:17:49.534422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.531637    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.565329 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.565298    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"]
Oct 30 22:17:49.569694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.569655    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.573746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.573717    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"]
Oct 30 22:17:49.624962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.624932    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.624962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.624966    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dg4v\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.624991    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.625168    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625321 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.625299    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625413 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.625373    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625481 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.625462    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625526 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.625491    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.625644 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.625622    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.657507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.657464    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"]
Oct 30 22:17:49.726467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726434    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.726640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726478    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.726640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726499    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cnw9\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.726640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726518    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.726640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726618    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.726852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726645    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dg4v\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.726852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726667    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.726852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726710    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.726852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726791    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.726852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726819    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.727094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726893    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.727094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726935    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.726970    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.727094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727000    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727071    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727094    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727163    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727215    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727227    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.727992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727549    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.727992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.727954    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.728207 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.728173    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.728364 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.728342    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.729177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.729156    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.729580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.729562    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.789334 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.789269    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.789559 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.789542    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dg4v\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.828575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828430    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.828776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828612    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.828776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828655    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.828776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828691    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.828776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828721    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cnw9\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.828776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828755    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.829387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828803    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.829387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828809    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.829387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828905    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.829387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.828996    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.829387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.829085    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.829387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.829191    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.830521 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.829544    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.830521 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.829815    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.831783 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.831763    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.832091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.832070    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.861306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.861278    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:49.891873 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.891843    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.892093 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.892075    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cnw9\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-vb457\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:49.894702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.894682    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:49.900095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:49.900073    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:50.030729 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.030699    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm\" (UniqueName: \"kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm\") pod \"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0\" (UID: \"8ce58bb5-442d-4dc3-b28e-6e7ef9efead0\") "
Oct 30 22:17:50.032924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.032896    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm" pod "8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" (UID: "8ce58bb5-442d-4dc3-b28e-6e7ef9efead0"). InnerVolumeSpecName "kube-api-access-rbmwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:50.133103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.133027    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbmwm\" (UniqueName: \"kubernetes.io/projected/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0-kube-api-access-rbmwm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:50.305170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.305125    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"]
Oct 30 22:17:50.305480 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:50.305455    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7ee2b3b_0d0b_4c7c_8bff_d0b50f72f6e0.slice/crio-88607ee33a09d80db1f5c060abafac08fce3270f160aa8007489fb4e7d2fd375 WatchSource:0}: Error finding container 88607ee33a09d80db1f5c060abafac08fce3270f160aa8007489fb4e7d2fd375: Status 404 returned error can't find the container with id 88607ee33a09d80db1f5c060abafac08fce3270f160aa8007489fb4e7d2fd375
Oct 30 22:17:50.307472 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:17:50.307386    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadb92766_aef3_4e47_bd55_0040f9090661.slice/crio-45dc654622d86d82fece24950b48fe59e67f5e7c1d87cb889ef2b3c47ee95265 WatchSource:0}: Error finding container 45dc654622d86d82fece24950b48fe59e67f5e7c1d87cb889ef2b3c47ee95265: Status 404 returned error can't find the container with id 45dc654622d86d82fece24950b48fe59e67f5e7c1d87cb889ef2b3c47ee95265
Oct 30 22:17:50.308807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.308769    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:50.308911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.308852    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:50.308911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.308895    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:50.310287 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.310253    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:50.310351 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.310327    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:50.310431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.310367    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:50.352323 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.352271    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"]
Oct 30 22:17:50.452667 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.452632    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-134d62"]
Oct 30 22:17:50.462012 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.461986    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-134d62"]
Oct 30 22:17:50.591487 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.591452    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"]
Oct 30 22:17:50.592339 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.592317    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" containerName="test-curl-7426-134d62"
Oct 30 22:17:50.592419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.592345    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" containerName="test-curl-7426-134d62"
Oct 30 22:17:50.592584 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.592568    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" containerName="test-curl-7426-134d62"
Oct 30 22:17:50.596691 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.596667    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.670264 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.670234    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"]
Oct 30 22:17:50.720437 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.720358    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" event={"ID":"adb92766-aef3-4e47-bd55-0040f9090661","Type":"ContainerStarted","Data":"433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57"}
Oct 30 22:17:50.720966 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.720940    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" event={"ID":"adb92766-aef3-4e47-bd55-0040f9090661","Type":"ContainerStarted","Data":"45dc654622d86d82fece24950b48fe59e67f5e7c1d87cb889ef2b3c47ee95265"}
Oct 30 22:17:50.726709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.726675    4536 generic.go:356] "Generic (PLEG): container finished" podID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" containerID="97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387" exitCode=137
Oct 30 22:17:50.726839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.726816    4536 scope.go:122] "RemoveContainer" containerID="97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387"
Oct 30 22:17:50.727005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.726986    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:50.734056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.734032    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" event={"ID":"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0","Type":"ContainerStarted","Data":"9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798"}
Oct 30 22:17:50.734226 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.734103    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7426-134d62" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" containerName="test-curl-7426-134d62" containerID="cri-o://399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b" gracePeriod=2
Oct 30 22:17:50.734347 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.734325    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" event={"ID":"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0","Type":"ContainerStarted","Data":"88607ee33a09d80db1f5c060abafac08fce3270f160aa8007489fb4e7d2fd375"}
Oct 30 22:17:50.741710 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741686    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.741847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741735    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.741847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741759    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.741847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741815    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4l8q\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.742107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741870    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.742107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741915    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.742107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.741931    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.742107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.742028    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.742107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.742050    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.746215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.746188    4536 scope.go:122] "RemoveContainer" containerID="97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387"
Oct 30 22:17:50.746672 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:50.746650    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387\": container with ID starting with 97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387 not found: ID does not exist" containerID="97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387"
Oct 30 22:17:50.746752 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.746684    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387"} err="failed to get container status \"97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387\": rpc error: code = NotFound desc = could not find container \"97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387\": container with ID starting with 97aadb26c967295a3da0dead01965d943569bfc576827fbbae779e2861e38387 not found: ID does not exist"
Oct 30 22:17:50.789825 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:50.789790    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-134d62\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:50.808030 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:50.807988    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:17:50.842983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.842951    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.842983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.842988    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.843238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.843114    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4l8q\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.843238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.843211    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.843351 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.843293    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.843351 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.843323    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.843484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.843423    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844060    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844098    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844178    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844309    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844489    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844868    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.845021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.844992    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.851433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.851371    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.851609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.851592    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.861510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.861370    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:50.909848 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.901494    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:50.934059 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.927977    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podStartSLOduration=1.927957863 podStartE2EDuration="1.927957863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:50.924961616 +0000 UTC m=+1401.912769880" watchObservedRunningTime="2026-10-30 22:17:50.927957863 +0000 UTC m=+1401.915766130"
Oct 30 22:17:50.942256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.941979    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4l8q\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:50.942256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:50.941993    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token\") pod \"e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") " pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:51.060919 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.060822    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podStartSLOduration=2.06080443 podStartE2EDuration="2.06080443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:51.057104476 +0000 UTC m=+1402.044912739" watchObservedRunningTime="2026-10-30 22:17:51.06080443 +0000 UTC m=+1402.048612694"
Oct 30 22:17:51.117853 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:51.117802    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-134d62\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:51.165590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.165557    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" path="/var/lib/kubelet/pods/8ce58bb5-442d-4dc3-b28e-6e7ef9efead0/volumes"
Oct 30 22:17:51.169270 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:51.169242    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-809de1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="8ce58bb5-442d-4dc3-b28e-6e7ef9efead0" pod="opendatahub/test-curl-7426-809de1"
Oct 30 22:17:51.216204 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.215747    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:51.685746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.685555    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"]
Oct 30 22:17:51.715125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.714303    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:51.715125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.714419    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:51.715125 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.714467    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:17:51.766472 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.766436    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" event={"ID":"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7","Type":"ContainerStarted","Data":"685b46ba75265d9774fa0b19db5bedad64cc20c38c74df3ad6cbb678afc74120"}
Oct 30 22:17:51.863288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.863246    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:51.863443 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.863305    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:51.911489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.911453    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:51.911724 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:51.911698    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:52.882170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:52.880006    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:52.882170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:52.880088    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:52.913181 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:52.913150    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:52.919552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:52.919506    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:52.951942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:52.941337    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" event={"ID":"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7","Type":"ContainerStarted","Data":"4dc8eb11653e76e7b6ed2f82b565f67b47320498b0c3af05f729466a63242501"}
Oct 30 22:17:53.010090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.008145    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Oct 30 22:17:53.037668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.037492    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:17:53.101456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.099086    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podStartSLOduration=3.099064159 podStartE2EDuration="3.099064159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:53.089957844 +0000 UTC m=+1404.077766122" watchObservedRunningTime="2026-10-30 22:17:53.099064159 +0000 UTC m=+1404.086872423"
Oct 30 22:17:53.237574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.226115    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:17:53.237574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.226236    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:53.237574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.226285    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:17:53.653431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.649977    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:53.756983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.756942    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr\" (UniqueName: \"kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr\") pod \"e64343e3-73c3-47d6-9bfc-c964ede522b4\" (UID: \"e64343e3-73c3-47d6-9bfc-c964ede522b4\") "
Oct 30 22:17:53.762016 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.761981    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr" pod "e64343e3-73c3-47d6-9bfc-c964ede522b4" (UID: "e64343e3-73c3-47d6-9bfc-c964ede522b4"). InnerVolumeSpecName "kube-api-access-ttncr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:53.860478 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.859952    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttncr\" (UniqueName: \"kubernetes.io/projected/e64343e3-73c3-47d6-9bfc-c964ede522b4-kube-api-access-ttncr\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:53.863491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.862847    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:53.863491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.862906    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:53.904422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.903809    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:53.904422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.903871    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:53.969422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.968673    4536 generic.go:356] "Generic (PLEG): container finished" podID="e64343e3-73c3-47d6-9bfc-c964ede522b4" containerID="399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b" exitCode=137
Oct 30 22:17:53.973422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.969676    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:53.979424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:53.973724    4536 scope.go:122] "RemoveContainer" containerID="399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b"
Oct 30 22:17:54.031785 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.031761    4536 scope.go:122] "RemoveContainer" containerID="399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b"
Oct 30 22:17:54.032522 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:54.032481    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b\": container with ID starting with 399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b not found: ID does not exist" containerID="399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b"
Oct 30 22:17:54.032722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.032695    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b"} err="failed to get container status \"399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b\": rpc error: code = NotFound desc = could not find container \"399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b\": container with ID starting with 399d4291b70b78edbf4efeeee5b58cdfa6ce38098d63c9b5d47951255912df3b not found: ID does not exist"
Oct 30 22:17:54.045701 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:17:54.045652    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7426-134d62\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" pod="opendatahub/test-curl-7426-134d62"
Oct 30 22:17:54.216966 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.216533    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:54.216966 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.216594    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:17:54.862300 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.862262    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:54.862509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.862314    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:54.903421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.902692    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:54.903421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:54.902750    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:55.222428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.219212    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:55.240666 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.222989    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:17:55.240666 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.231443    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e64343e3-73c3-47d6-9bfc-c964ede522b4" path="/var/lib/kubelet/pods/e64343e3-73c3-47d6-9bfc-c964ede522b4/volumes"
Oct 30 22:17:55.861956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.861925    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:55.862203 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.862171    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:55.921470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.921431    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:55.921741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:55.921713    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:56.217367 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:56.217335    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:56.217670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:56.217626    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:17:56.861995 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:56.861962    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:56.862496 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:56.862468    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:56.902853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:56.902814    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:56.903178 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:56.903149    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:57.216614 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:57.216587    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:57.216849 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:57.216829    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:17:57.862759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:57.862720    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:17:57.863179 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:57.862802    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:17:57.904424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:57.903798    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:17:57.904424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:57.903864    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:17:58.216509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.216473    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:58.216788 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.216759    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:17:58.866755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.866727    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:58.867203 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.867036    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:58.867784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.867768    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:17:58.905933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.905904    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:58.906359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.906331    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:58.907299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:58.907281    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:17:59.219936 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:59.219904    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:17:59.220157 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:17:59.219966    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:18:00.216411 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:00.216367    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 22:18:00.216832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:00.216428    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 22:18:01.226220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:01.226186    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:18:01.226743 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:01.226699    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:18:01.227744 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:01.227720    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:18:02.694537 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.694503    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"]
Oct 30 22:18:02.699633 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.699609    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.755918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.755883    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4266be31-openshift-default-dockercfg-wfm9n\""
Oct 30 22:18:02.775773 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.775739    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.775979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.775958    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthc5\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776055    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776150 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776112    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776308 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776289    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776389 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776367    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776564 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776538    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776620    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.776686 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.776676    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.799613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.799564    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"]
Oct 30 22:18:02.880634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.880597    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.881916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.881891    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.882520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.882498    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.883019 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.882995    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.883722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.883701    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.890847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.890818    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.891602 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.891581    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthc5\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.892114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.892093    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.892636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.892619    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.893259 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.881792    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.893417 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.883430    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.893530 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.882904    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.893644 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.892547    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.893759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.884104    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.897081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.897058    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.933418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.932680    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.965483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.959166    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:02.979193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:02.979158    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthc5\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5\") pod \"e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") " pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:03.015002 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:03.014684    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:03.618580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:03.617603    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:03.618580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:03.617692    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:03.618580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:03.617731    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:03.619523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:03.619498    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"]
Oct 30 22:18:04.353183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:04.353145    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" event={"ID":"e3619c04-3858-4435-b51a-91b28b1a2265","Type":"ContainerStarted","Data":"321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"}
Oct 30 22:18:04.353714 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:04.353212    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" event={"ID":"e3619c04-3858-4435-b51a-91b28b1a2265","Type":"ContainerStarted","Data":"7e3a2fa66f65f44c9c1925510f48096483bf6ef0287d4337fb437230e9d210df"}
Oct 30 22:18:04.483212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:04.483143    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podStartSLOduration=2.483116441 podStartE2EDuration="2.483116441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:04.478510678 +0000 UTC m=+1415.466318957" watchObservedRunningTime="2026-10-30 22:18:04.483116441 +0000 UTC m=+1415.470924705"
Oct 30 22:18:05.016302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.016272    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:05.022467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.022437    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:05.359776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.359688    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:05.360885 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.360867    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:18:05.413496 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.413464    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"]
Oct 30 22:18:05.417655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.417633    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.517023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.516975    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"]
Oct 30 22:18:05.554101 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554055    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554101 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554099    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554320 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554127    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b46s8\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554512 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554476    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554600 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554532    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554707    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554740    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554833    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.554901 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.554862    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656554 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656452    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656554 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656493    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656595    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656622    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656764    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656793    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656819    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b46s8\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656850    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.656948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.656879    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.657152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.657050    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.657209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.657191    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.657346 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.657324    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.657441 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.657330    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.657511 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.657490    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.659453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.659427    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.659753 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.659733    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.719712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.719672    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b46s8\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.719863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.719774    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:05.736467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:05.736442    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:06.181491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.181461    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"]
Oct 30 22:18:06.183449 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:18:06.183418    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e049f4_7335_4ce8_8fab_e8dfee377277.slice/crio-1eef3191c77a8c0320a5f8cd83f3b1117ae405da16832b883f562b4db753c889 WatchSource:0}: Error finding container 1eef3191c77a8c0320a5f8cd83f3b1117ae405da16832b883f562b4db753c889: Status 404 returned error can't find the container with id 1eef3191c77a8c0320a5f8cd83f3b1117ae405da16832b883f562b4db753c889
Oct 30 22:18:06.186152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.186115    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:06.186266 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.186201    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:06.186336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.186273    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:06.369012 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.368978    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" event={"ID":"06e049f4-7335-4ce8-8fab-e8dfee377277","Type":"ContainerStarted","Data":"b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4"}
Oct 30 22:18:06.369476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.369040    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" event={"ID":"06e049f4-7335-4ce8-8fab-e8dfee377277","Type":"ContainerStarted","Data":"1eef3191c77a8c0320a5f8cd83f3b1117ae405da16832b883f562b4db753c889"}
Oct 30 22:18:06.496760 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.496697    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podStartSLOduration=1.496680935 podStartE2EDuration="1.496680935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:06.49405215 +0000 UTC m=+1417.481860414" watchObservedRunningTime="2026-10-30 22:18:06.496680935 +0000 UTC m=+1417.484489256"
Oct 30 22:18:06.737014 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:06.736984    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:07.743974 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:07.743934    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:07.744590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:07.744020    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:08.737246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:08.737213    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:08.737438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:08.737266    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:09.748423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:09.746731    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:09.748423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:09.747043    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:10.740008 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:10.739964    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:10.740301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:10.740271    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:10.987638 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:10.986494    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Oct 30 22:18:11.152606 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.152498    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:11.199424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.193490    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:11.316558 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.301485    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:11.487777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.487714    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:11.609575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.609535    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:11.768473 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.746067    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:11.768473 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:11.768227    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:12.009421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.006522    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:12.009421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.007386    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:18:12.047522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.034477    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:12.161749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.161698    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 30 22:18:12.178108 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.178079    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:18:12.250441 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.201864    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:12.250441 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.220559    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:18:12.326497 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.303521    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 30 22:18:12.326497 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.308569    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:18:12.650878 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.649212    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": context deadline exceeded"
Oct 30 22:18:12.650878 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.649547    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:18:12.650878 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.649921    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:18:12.735718 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.735655    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": context deadline exceeded"
Oct 30 22:18:12.747619 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.747581    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:12.788703 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.788669    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:12.788964 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.786880    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:12.789485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:12.789467    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:18:13.011687 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.011644    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.041659 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.041601    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.041982 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.041954    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.042604 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.042586    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:18:13.218437 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.210631    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.218437 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.213912    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.341485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.315941    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.655908 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.655794    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:13.883522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.800544    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:13.883522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.800608    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:13.883522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.801175    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Oct 30 22:18:13.964619 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.953033    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.001319 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:13.992958    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:18:14.026426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.023939    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.026426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.024417    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.043527 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.043486    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.263561 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.263455    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.343438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.323258    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.435464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.346673    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.527752 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.527659    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.565641 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.565592    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:18:14.573474 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.569481    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:14.682563 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.682513    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:18:14.808621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.808517    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:14.842597 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:14.842552    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:15.004847 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.004788    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:15.034458 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.022098    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:18:15.034458 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.022161    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:15.039437 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.038062    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:15.069515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.068447    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:15.267971 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.267931    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": context deadline exceeded"
Oct 30 22:18:15.346499 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.346372    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:15.371948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.371899    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 30 22:18:15.704783 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.692968    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:15.773608 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.773568    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:15.773892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:15.773859    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:16.150651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.150545    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Oct 30 22:18:16.177425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.152058    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Oct 30 22:18:16.181423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.178069    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:16.207017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.206985    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:18:16.305420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.294186    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:16.375424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.350635    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 30 22:18:16.378420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.378359    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:16.485450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.485375    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:16.726417 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.726276    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:16.834539 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.818526    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:16.834799 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.834758    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:16.931425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:16.911959    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:18:17.177979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.177866    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Oct 30 22:18:17.179443 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.179389    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Oct 30 22:18:17.258426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.207671    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:17.319423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.310517    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:17.420607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.369630    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Oct 30 22:18:17.420607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.379880    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:17.500422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.456230    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:18:17.765468 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.764501    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:17.765468 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.764562    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:17.848887 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.848857    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:18:17.931889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:17.931851    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:18:18.021465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:18.016144    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:18:18.039118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:18.039091    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:18:18.046486 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:18.046462    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:18:18.739037 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:18.738999    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 22:18:18.749718 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:18.749662    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 22:18:20.009159 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:20.009129    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:20.010595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:20.010566    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:20.011086 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:20.011070    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:18:20.810168 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:18:20.810131    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:18:29.322127 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.319522    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"]
Oct 30 22:18:29.346788 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.346744    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"]
Oct 30 22:18:29.346987 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.346871    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.426930 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.426833    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume\") pod \"payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") " pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.427448 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.427423    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7cb\" (UniqueName: \"kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb\") pod \"payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") " pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.462719 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.462689    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4266be31\""
Oct 30 22:18:29.462897 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.462748    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4266be31-dockercfg-5d9mk\""
Oct 30 22:18:29.528808 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.528767    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume\") pod \"payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") " pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.529023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.528956    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7cb\" (UniqueName: \"kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb\") pod \"payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") " pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.529551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.529525    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume\") pod \"payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") " pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.591325 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.591260    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7cb\" (UniqueName: \"kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb\") pod \"payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") " pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:29.750030 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:29.749990    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:30.128464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:30.128442    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"]
Oct 30 22:18:30.129470 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:18:30.129443    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c61af7_fbf8_4ed1_8eaa_55756cd66eec.slice/crio-afb66e6bafbc08fe19a3ab6c57a4627403fc26d0e286c082b96d80ecffa05114 WatchSource:0}: Error finding container afb66e6bafbc08fe19a3ab6c57a4627403fc26d0e286c082b96d80ecffa05114: Status 404 returned error can't find the container with id afb66e6bafbc08fe19a3ab6c57a4627403fc26d0e286c082b96d80ecffa05114
Oct 30 22:18:31.128538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:31.128496    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb" event={"ID":"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec","Type":"ContainerStarted","Data":"afb66e6bafbc08fe19a3ab6c57a4627403fc26d0e286c082b96d80ecffa05114"}
Oct 30 22:18:33.142061 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:33.142026    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb" event={"ID":"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec","Type":"ContainerStarted","Data":"3456df28accc89e7df67b5d228308c5649af73204825b1c075af73bb9794ffad"}
Oct 30 22:18:33.142581 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:33.142152    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:33.297907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:33.297813    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb" podStartSLOduration=1.473826194 podStartE2EDuration="4.297801445s" totalImagesPullingTime="2.823975251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:33.265328018 +0000 UTC m=+1444.253136283" watchObservedRunningTime="2026-10-30 22:18:33.297801445 +0000 UTC m=+1444.285609738"
Oct 30 22:18:34.865599 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:34.865561    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"]
Oct 30 22:18:34.870209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:34.870187    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:34.977148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:34.977115    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"]
Oct 30 22:18:35.023274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023239    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023422    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpr4k\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023460    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023517    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023535    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023558    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023595    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023793 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023633    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.023793 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.023660    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124359    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpr4k\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124430    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124519    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124552    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124584    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124642    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124786    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124843    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124977 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124909    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.124977 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.124846    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.125085 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.125053    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.125148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.125125    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.125256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.125239    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.125322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.125300    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.126945 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.126917    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.127303 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.127285    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.186645 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.186618    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.186885 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.186866    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpr4k\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k\") pod \"e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") " pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.483968 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.483938    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:35.644630 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.644596    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"]
Oct 30 22:18:35.650136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.650112    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.712508 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.712482    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"]
Oct 30 22:18:35.733005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.732975    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733185 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733013    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733185 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733044    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgjx\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733185 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733105    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733354 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733212    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733367    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733504    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733559    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.733681 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.733665    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835040    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835089    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835148    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835225    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835253    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835289    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgjx\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835310    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835698 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835360    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835698 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835476    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835799 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835704    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835854 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835813    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.835854 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835817    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.836126 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.835951    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.836126 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.836052    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.837642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.837613    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.838003 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.837982    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.897645 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.897616    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.897965 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.897923    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgjx\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx\") pod \"e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") " pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:35.939810 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.939787    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"]
Oct 30 22:18:35.940905 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:18:35.940881    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c901e3_b33d_4573_b99f_42929b91502e.slice/crio-d81e7091e21b9c3a6c57c8849f205ac276d98c9bff21cc7b3d00c1e7018651c4 WatchSource:0}: Error finding container d81e7091e21b9c3a6c57c8849f205ac276d98c9bff21cc7b3d00c1e7018651c4: Status 404 returned error can't find the container with id d81e7091e21b9c3a6c57c8849f205ac276d98c9bff21cc7b3d00c1e7018651c4
Oct 30 22:18:35.943354 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.943321    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:35.943440 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.943424    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:35.943477 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.943460    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:35.964359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:35.964339    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:36.161296 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.161212    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" event={"ID":"63c901e3-b33d-4573-b99f-42929b91502e","Type":"ContainerStarted","Data":"ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f"}
Oct 30 22:18:36.161296 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.161264    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" event={"ID":"63c901e3-b33d-4573-b99f-42929b91502e","Type":"ContainerStarted","Data":"d81e7091e21b9c3a6c57c8849f205ac276d98c9bff21cc7b3d00c1e7018651c4"}
Oct 30 22:18:36.285990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.285927    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podStartSLOduration=2.285908026 podStartE2EDuration="2.285908026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:36.283546193 +0000 UTC m=+1447.271354459" watchObservedRunningTime="2026-10-30 22:18:36.285908026 +0000 UTC m=+1447.273716291"
Oct 30 22:18:36.398689 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.398659    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"]
Oct 30 22:18:36.399202 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:18:36.399174    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a943de_9159_4e5b_85cb_f2ff60788d05.slice/crio-0b579b687e5348a8c758c354d1c593a5de857d4f375bcc86bb26f86668e6ead1 WatchSource:0}: Error finding container 0b579b687e5348a8c758c354d1c593a5de857d4f375bcc86bb26f86668e6ead1: Status 404 returned error can't find the container with id 0b579b687e5348a8c758c354d1c593a5de857d4f375bcc86bb26f86668e6ead1
Oct 30 22:18:36.402090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.402051    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:36.402164 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.402140    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:36.402202 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.402185    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:18:36.484483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:36.484454    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:37.170306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.170278    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" event={"ID":"99a943de-9159-4e5b-85cb-f2ff60788d05","Type":"ContainerStarted","Data":"3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3"}
Oct 30 22:18:37.170712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.170320    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" event={"ID":"99a943de-9159-4e5b-85cb-f2ff60788d05","Type":"ContainerStarted","Data":"0b579b687e5348a8c758c354d1c593a5de857d4f375bcc86bb26f86668e6ead1"}
Oct 30 22:18:37.360187 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.360134    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podStartSLOduration=2.36011954 podStartE2EDuration="2.36011954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:37.358748175 +0000 UTC m=+1448.346556439" watchObservedRunningTime="2026-10-30 22:18:37.36011954 +0000 UTC m=+1448.347927802"
Oct 30 22:18:37.491452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.488593    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:18:37.491452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.488719    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:18:37.965373 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.965346    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:37.966448 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.966427    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 22:18:37.966517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:37.966475    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 22:18:38.499517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:38.499482    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:18:38.500117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:38.500083    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:18:38.965223 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:38.965187    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 22:18:38.965434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:38.965241    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 22:18:39.497422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:39.496673    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:18:39.497422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:39.496734    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:18:39.965317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:39.965291    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 22:18:39.965821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:39.965491    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 22:18:40.490829 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:40.490792    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:18:40.491150 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:40.491112    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:18:40.965607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:40.965575    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 22:18:40.966077 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:40.965632    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 22:18:41.492422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:41.491838    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:18:41.492422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:41.491903    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:18:41.964892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:41.964840    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 22:18:41.965199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:41.965171    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 22:18:42.489799 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:42.489762    4536 patch_prober.go:29] interesting pod/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 22:18:42.490427 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:42.490371    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 22:18:42.965451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:42.965416    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 22:18:42.965649 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:42.965475    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 22:18:42.982541 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:42.982506    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"]
Oct 30 22:18:42.988161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:42.988141    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.081214 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.081175    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-73fb6577-serving-cert\""
Oct 30 22:18:43.087021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.086851    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"]
Oct 30 22:18:43.131162 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.131124    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.131480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.131454    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.131688 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.131670    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrs97\" (UniqueName: \"kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.132092 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.132072    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.261467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.248356    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.261467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.248422    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.261467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.248451    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrs97\" (UniqueName: \"kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.261467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.248742    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.301422 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:18:43.265618    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-serving-cert: secret "maas-api-e2e-ait-73fb6577-serving-cert" not found
Oct 30 22:18:43.301422 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:18:43.265730    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls podName:7e186f81-bf56-4219-85e8-478b9c77015b nodeName:}" failed. No retries permitted until 2026-10-30 22:18:43.765705529 +0000 UTC m=+1454.753513790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls") pod "maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" (UID: "7e186f81-bf56-4219-85e8-478b9c77015b") : secret "maas-api-e2e-ait-73fb6577-serving-cert" not found
Oct 30 22:18:43.301422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.268266    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.301422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.277370    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.377763 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.377730    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrs97\" (UniqueName: \"kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.759832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.759786    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:43.760712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.760649    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:43.765289 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.765270    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:18:43.866820 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.866791    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.870207 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.870180    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\") pod \"maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:43.903405 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:43.903373    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:44.147184 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:44.147106    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:44.149138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:44.148201    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:44.150689 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:44.150664    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:18:44.151696 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:44.151674    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:18:44.293716 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:44.293693    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"]
Oct 30 22:18:44.294884 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:18:44.294861    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e186f81_bf56_4219_85e8_478b9c77015b.slice/crio-875937a02526f08dab8b58495140fe7203b6b3e00bce5a0ea2ef03646bcc58bc WatchSource:0}: Error finding container 875937a02526f08dab8b58495140fe7203b6b3e00bce5a0ea2ef03646bcc58bc: Status 404 returned error can't find the container with id 875937a02526f08dab8b58495140fe7203b6b3e00bce5a0ea2ef03646bcc58bc
Oct 30 22:18:44.429779 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:44.429689    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" event={"ID":"7e186f81-bf56-4219-85e8-478b9c77015b","Type":"ContainerStarted","Data":"875937a02526f08dab8b58495140fe7203b6b3e00bce5a0ea2ef03646bcc58bc"}
Oct 30 22:18:45.457636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:45.457600    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" event={"ID":"7e186f81-bf56-4219-85e8-478b9c77015b","Type":"ContainerStarted","Data":"91c272f5c91a2ecdd51e82277ff7ea87931f551ec2bcf3dac66aa70597840694"}
Oct 30 22:18:45.458876 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:45.458852    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:45.591549 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:45.591485    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" podStartSLOduration=2.768198446 podStartE2EDuration="3.591465484s" totalImagesPullingTime="823.267038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:45.587478876 +0000 UTC m=+1456.575287140" watchObservedRunningTime="2026-10-30 22:18:45.591465484 +0000 UTC m=+1456.579273748"
Oct 30 22:18:50.929341 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:18:50.929296    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:18:52.027420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.013496    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:52.240821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.239501    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:52.264421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.263489    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:52.365596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.365490    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:52.513421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.511112    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:18:52.688639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.685507    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:52.721701 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:52.721597    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:53.043925 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:53.043882    4536 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:18:59.344650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.344613    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"]
Oct 30 22:18:59.350206 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.350177    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.408220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.408184    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"]
Oct 30 22:18:59.452271 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.452237    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ea4ac3ec-serving-cert\""
Oct 30 22:18:59.471221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.471196    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.471432 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.471382    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.471516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.471500    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9skm5\" (UniqueName: \"kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.471554 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.471542    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.572649 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.572618    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.572833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.572690    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9skm5\" (UniqueName: \"kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.572833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.572720    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.572833 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:18:59.572771    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-serving-cert: secret "maas-api-e2e-ait-ea4ac3ec-serving-cert" not found
Oct 30 22:18:59.572833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.572788    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.572833 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:18:59.572827    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls podName:e78f1ffe-1463-4430-ae72-919affad054a nodeName:}" failed. No retries permitted until 2026-10-30 22:19:00.072812225 +0000 UTC m=+1471.060620466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls") pod "maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" (UID: "e78f1ffe-1463-4430-ae72-919affad054a") : secret "maas-api-e2e-ait-ea4ac3ec-serving-cert" not found
Oct 30 22:18:59.573598 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.573577    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.575238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.575218    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:18:59.658257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:18:59.658053    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9skm5\" (UniqueName: \"kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:19:00.103614 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:00.103582    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:19:00.106231 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:00.106207    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\") pod \"maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:19:00.268027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:00.267996    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:19:00.652064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:00.652039    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"]
Oct 30 22:19:00.652553 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:19:00.652512    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78f1ffe_1463_4430_ae72_919affad054a.slice/crio-9c491b6533175b0a3ff1a13774d0a77c51507fe1676bbb1d67fc00dd97216008 WatchSource:0}: Error finding container 9c491b6533175b0a3ff1a13774d0a77c51507fe1676bbb1d67fc00dd97216008: Status 404 returned error can't find the container with id 9c491b6533175b0a3ff1a13774d0a77c51507fe1676bbb1d67fc00dd97216008
Oct 30 22:19:00.739285 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:00.739231    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" event={"ID":"e78f1ffe-1463-4430-ae72-919affad054a","Type":"ContainerStarted","Data":"9c491b6533175b0a3ff1a13774d0a77c51507fe1676bbb1d67fc00dd97216008"}
Oct 30 22:19:01.748131 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:01.747947    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" event={"ID":"e78f1ffe-1463-4430-ae72-919affad054a","Type":"ContainerStarted","Data":"755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be"}
Oct 30 22:19:01.748131 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:01.748023    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:19:01.869932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:01.869867    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" podStartSLOduration=2.039374901 podStartE2EDuration="2.869846264s" totalImagesPullingTime="830.471363ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:01.868497341 +0000 UTC m=+1472.856305607" watchObservedRunningTime="2026-10-30 22:19:01.869846264 +0000 UTC m=+1472.857654528"
Oct 30 22:19:04.556678 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.556640    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"]
Oct 30 22:19:04.561199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.561175    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.576670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.576641    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.576806 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.576695    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.576806 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.576792    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krb9m\" (UniqueName: \"kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.576950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.576931    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.614686 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.614657    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4266be31-serving-cert\""
Oct 30 22:19:04.668973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.668939    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"]
Oct 30 22:19:04.679336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.679304    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.679560 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.679541    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.679634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.679595    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.679693 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.679656    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krb9m\" (UniqueName: \"kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.679775 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:19:04.679752    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-serving-cert: secret "maas-api-e2e-ait-4266be31-serving-cert" not found
Oct 30 22:19:04.679842 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:19:04.679829    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls podName:1f013cb4-6056-4acf-8fb1-10df58a5ce21 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:05.179807827 +0000 UTC m=+1476.167616082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls") pod "maas-api-e2e-ait-4266be31-6444556b-lfmdj" (UID: "1f013cb4-6056-4acf-8fb1-10df58a5ce21") : secret "maas-api-e2e-ait-4266be31-serving-cert" not found
Oct 30 22:19:04.680248 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.680230    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.682014 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.681992    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:04.742990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:04.742962    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krb9m\" (UniqueName: \"kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:05.190362 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:05.190328    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:05.192938 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:05.192912    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\") pod \"maas-api-e2e-ait-4266be31-6444556b-lfmdj\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:05.474716 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:05.474684    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:05.860911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:05.860884    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"]
Oct 30 22:19:05.861364 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:19:05.861239    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f013cb4_6056_4acf_8fb1_10df58a5ce21.slice/crio-5227fcd0d8d793997f0a0bb7e77b6095e0ed949c0e40a4a7f399161185fe0191 WatchSource:0}: Error finding container 5227fcd0d8d793997f0a0bb7e77b6095e0ed949c0e40a4a7f399161185fe0191: Status 404 returned error can't find the container with id 5227fcd0d8d793997f0a0bb7e77b6095e0ed949c0e40a4a7f399161185fe0191
Oct 30 22:19:06.785124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:06.785087    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" event={"ID":"1f013cb4-6056-4acf-8fb1-10df58a5ce21","Type":"ContainerStarted","Data":"b77d22653ca72e717ee514325478c0d7192a6e62caa2084c02f0faed6eef4c8a"}
Oct 30 22:19:06.785302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:06.785136    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" event={"ID":"1f013cb4-6056-4acf-8fb1-10df58a5ce21","Type":"ContainerStarted","Data":"5227fcd0d8d793997f0a0bb7e77b6095e0ed949c0e40a4a7f399161185fe0191"}
Oct 30 22:19:06.785302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:06.785236    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:06.922490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:06.915660    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" podStartSLOduration=2.117363158 podStartE2EDuration="2.915637694s" totalImagesPullingTime="798.274536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:06.911470697 +0000 UTC m=+1477.899278957" watchObservedRunningTime="2026-10-30 22:19:06.915637694 +0000 UTC m=+1477.903445958"
Oct 30 22:19:07.764469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:07.764442    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:19:12.798273 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:12.798196    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:19:20.806191 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:19:20.806152    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:19:22.461091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.461055    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"]
Oct 30 22:19:22.464161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.464142    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.516961 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.516934    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9qrfj\""
Oct 30 22:19:22.551182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.551143    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.551574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.551550    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.551786 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.551769    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.551971 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.551954    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.552063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.551984    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9rkp\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.552122 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.552104    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.552194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.552175    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.552309 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.552292    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.552505 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.552482    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.577469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.577435    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"]
Oct 30 22:19:22.654095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654060    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654303 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654095    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654474 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654460    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654530    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654624 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654606    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654729 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654701    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654804    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654963 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654843    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9rkp\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.654963 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654943    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.655080 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.654992    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.665270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.655202    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.665270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.655274    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.665270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.655548    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.665270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.655669    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.665270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.657857    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.665270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.658122    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.718167 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.718100    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.718319 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.718286    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9rkp\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp\") pod \"maas-default-gateway-openshift-default-76597d6855-mjr2j\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:22.777341 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:22.777311    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:23.234886 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:23.234864    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"]
Oct 30 22:19:23.235348 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:19:23.235322    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e5454b_4393_4649_aeb0_a58220823698.slice/crio-9d5b9cee719f900151916eea1c9bc92d9bf9709bc4d38ec29111af3d6e87e817 WatchSource:0}: Error finding container 9d5b9cee719f900151916eea1c9bc92d9bf9709bc4d38ec29111af3d6e87e817: Status 404 returned error can't find the container with id 9d5b9cee719f900151916eea1c9bc92d9bf9709bc4d38ec29111af3d6e87e817
Oct 30 22:19:23.237947 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:23.237913    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:19:23.238026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:23.237977    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:19:23.238026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:23.238011    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:19:23.903353 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:23.903313    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" event={"ID":"c2e5454b-4393-4649-aeb0-a58220823698","Type":"ContainerStarted","Data":"7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09"}
Oct 30 22:19:23.903903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:23.903381    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" event={"ID":"c2e5454b-4393-4649-aeb0-a58220823698","Type":"ContainerStarted","Data":"9d5b9cee719f900151916eea1c9bc92d9bf9709bc4d38ec29111af3d6e87e817"}
Oct 30 22:19:24.092428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:24.077492    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podStartSLOduration=2.077473796 podStartE2EDuration="2.077473796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:24.075443527 +0000 UTC m=+1495.063251790" watchObservedRunningTime="2026-10-30 22:19:24.077473796 +0000 UTC m=+1495.065282059"
Oct 30 22:19:24.786788 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:24.786755    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:24.792537 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:24.792499    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:24.792750 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:24.792723    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:25.796454 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:25.794503    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:25.796454 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:25.794567    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:26.789424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:26.786776    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:26.789424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:26.786840    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:27.781424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:27.777728    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:27.781424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:27.777799    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:28.782753 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:28.782715    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:28.783358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:28.783329    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:29.778619 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:29.778577    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:29.778788 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:29.778644    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:30.778538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:30.778508    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:19:30.778935 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:30.778559    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:19:31.782691 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:31.782663    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:31.783145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:31.782982    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:31.783794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:31.783776    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:19:45.250848 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:45.250817    4536 generic.go:356] "Generic (PLEG): container finished" podID="e3619c04-3858-4435-b51a-91b28b1a2265" containerID="321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643" exitCode=0
Oct 30 22:19:45.251294 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:45.250892    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" event={"ID":"e3619c04-3858-4435-b51a-91b28b1a2265","Type":"ContainerDied","Data":"321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"}
Oct 30 22:19:45.251372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:45.251359    4536 scope.go:122] "RemoveContainer" containerID="321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"
Oct 30 22:19:45.253091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:45.253061    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:19:45.253173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:45.253132    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:19:45.253173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:45.253165    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:19:46.259473 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:46.259440    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" event={"ID":"e3619c04-3858-4435-b51a-91b28b1a2265","Type":"ContainerStarted","Data":"d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6"}
Oct 30 22:19:47.016177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:47.016146    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:19:47.017166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:47.017143    4536 patch_prober.go:29] interesting pod/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:19:47.017285 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:47.017201    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:19:48.016295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:48.016263    4536 patch_prober.go:29] interesting pod/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:19:48.016873 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:48.016315    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:19:49.015919 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:49.015879    4536 patch_prober.go:29] interesting pod/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:19:49.016099 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:49.015935    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:19:50.016488 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:50.016451    4536 patch_prober.go:29] interesting pod/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:19:50.016877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:50.016533    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:19:50.359690 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:50.359607    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:19:50.828848 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:19:50.828810    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:19:51.016275 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:51.016239    4536 patch_prober.go:29] interesting pod/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 22:19:51.016481 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:51.016301    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 22:19:52.020244 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:52.020208    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:19:52.021467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:19:52.021445    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:20:20.806066 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:20:20.806032    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:20:22.438386 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:22.438356    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"]
Oct 30 22:20:27.439173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.439128    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb" podUID="b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" containerName="payload-processing" containerID="cri-o://3456df28accc89e7df67b5d228308c5649af73204825b1c075af73bb9794ffad" gracePeriod=25
Oct 30 22:20:27.644738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.644699    4536 generic.go:356] "Generic (PLEG): container finished" podID="b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" containerID="3456df28accc89e7df67b5d228308c5649af73204825b1c075af73bb9794ffad" exitCode=0
Oct 30 22:20:27.644911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.644743    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb" event={"ID":"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec","Type":"ContainerDied","Data":"3456df28accc89e7df67b5d228308c5649af73204825b1c075af73bb9794ffad"}
Oct 30 22:20:27.728602 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.728574    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:20:27.816615 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.816587    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume\") pod \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") "
Oct 30 22:20:27.816790 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.816695    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb\" (UniqueName: \"kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb\") pod \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\" (UID: \"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec\") "
Oct 30 22:20:27.817081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.817050    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume" pod "b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" (UID: "b3c61af7-fbf8-4ed1-8eaa-55756cd66eec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:27.817206 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.817190    4536 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-plugins-config-volume\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:27.818908 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.818882    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb" pod "b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" (UID: "b3c61af7-fbf8-4ed1-8eaa-55756cd66eec"). InnerVolumeSpecName "kube-api-access-hn7cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:27.918346 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:27.918311    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn7cb\" (UniqueName: \"kubernetes.io/projected/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec-kube-api-access-hn7cb\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:28.652051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:28.652022    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"
Oct 30 22:20:28.652586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:28.652069    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb" event={"ID":"b3c61af7-fbf8-4ed1-8eaa-55756cd66eec","Type":"ContainerDied","Data":"afb66e6bafbc08fe19a3ab6c57a4627403fc26d0e286c082b96d80ecffa05114"}
Oct 30 22:20:28.652586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:28.652135    4536 scope.go:122] "RemoveContainer" containerID="3456df28accc89e7df67b5d228308c5649af73204825b1c075af73bb9794ffad"
Oct 30 22:20:28.822241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:28.822194    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"]
Oct 30 22:20:28.869917 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:28.869886    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-4266be31-6f6c66dc7c-2s4vb"]
Oct 30 22:20:29.268441 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:29.268188    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" path="/var/lib/kubelet/pods/b3c61af7-fbf8-4ed1-8eaa-55756cd66eec/volumes"
Oct 30 22:20:38.457309 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:38.457270    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"]
Oct 30 22:20:38.457859 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:38.457537    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" podUID="7e186f81-bf56-4219-85e8-478b9c77015b" containerName="maas-api" containerID="cri-o://91c272f5c91a2ecdd51e82277ff7ea87931f551ec2bcf3dac66aa70597840694" gracePeriod=30
Oct 30 22:20:38.793332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:38.793245    4536 generic.go:356] "Generic (PLEG): container finished" podID="7e186f81-bf56-4219-85e8-478b9c77015b" containerID="91c272f5c91a2ecdd51e82277ff7ea87931f551ec2bcf3dac66aa70597840694" exitCode=0
Oct 30 22:20:38.793531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:38.793487    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" event={"ID":"7e186f81-bf56-4219-85e8-478b9c77015b","Type":"ContainerDied","Data":"91c272f5c91a2ecdd51e82277ff7ea87931f551ec2bcf3dac66aa70597840694"}
Oct 30 22:20:39.021732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.021691    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:20:39.072490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.062850    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca\") pod \"7e186f81-bf56-4219-85e8-478b9c77015b\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") "
Oct 30 22:20:39.072490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.062997    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs\") pod \"7e186f81-bf56-4219-85e8-478b9c77015b\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") "
Oct 30 22:20:39.072490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.063153    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\") pod \"7e186f81-bf56-4219-85e8-478b9c77015b\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") "
Oct 30 22:20:39.072490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.063300    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97\" (UniqueName: \"kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97\") pod \"7e186f81-bf56-4219-85e8-478b9c77015b\" (UID: \"7e186f81-bf56-4219-85e8-478b9c77015b\") "
Oct 30 22:20:39.081868 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.081804    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls" pod "7e186f81-bf56-4219-85e8-478b9c77015b" (UID: "7e186f81-bf56-4219-85e8-478b9c77015b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:39.086697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.086659    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca" pod "7e186f81-bf56-4219-85e8-478b9c77015b" (UID: "7e186f81-bf56-4219-85e8-478b9c77015b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:39.088580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.088544    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs" pod "7e186f81-bf56-4219-85e8-478b9c77015b" (UID: "7e186f81-bf56-4219-85e8-478b9c77015b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:39.110209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.108744    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97" pod "7e186f81-bf56-4219-85e8-478b9c77015b" (UID: "7e186f81-bf56-4219-85e8-478b9c77015b"). InnerVolumeSpecName "kube-api-access-zrs97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:39.187660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.180310    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.187935 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.187911    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrs97\" (UniqueName: \"kubernetes.io/projected/7e186f81-bf56-4219-85e8-478b9c77015b-kube-api-access-zrs97\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.188044 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.188031    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e186f81-bf56-4219-85e8-478b9c77015b-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.188134 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.188123    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e186f81-bf56-4219-85e8-478b9c77015b-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:39.806501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.806426    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"
Oct 30 22:20:39.806501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.806457    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr" event={"ID":"7e186f81-bf56-4219-85e8-478b9c77015b","Type":"ContainerDied","Data":"875937a02526f08dab8b58495140fe7203b6b3e00bce5a0ea2ef03646bcc58bc"}
Oct 30 22:20:39.806986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.806517    4536 scope.go:122] "RemoveContainer" containerID="91c272f5c91a2ecdd51e82277ff7ea87931f551ec2bcf3dac66aa70597840694"
Oct 30 22:20:39.978972 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:39.978941    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"]
Oct 30 22:20:40.028752 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:40.028657    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-73fb6577-78c7b89bbd-h9hsr"]
Oct 30 22:20:41.207551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:41.205662    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e186f81-bf56-4219-85e8-478b9c77015b" path="/var/lib/kubelet/pods/7e186f81-bf56-4219-85e8-478b9c77015b/volumes"
Oct 30 22:20:50.805712 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:20:50.805676    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:20:50.929651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:50.929617    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"]
Oct 30 22:20:50.929850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:50.929830    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" podUID="e78f1ffe-1463-4430-ae72-919affad054a" containerName="maas-api" containerID="cri-o://755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be" gracePeriod=30
Oct 30 22:20:51.235190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.234881    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:20:51.333450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.332055    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5\" (UniqueName: \"kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5\") pod \"e78f1ffe-1463-4430-ae72-919affad054a\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") "
Oct 30 22:20:51.333450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.332114    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\") pod \"e78f1ffe-1463-4430-ae72-919affad054a\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") "
Oct 30 22:20:51.333450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.332306    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca\") pod \"e78f1ffe-1463-4430-ae72-919affad054a\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") "
Oct 30 22:20:51.333450 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.332455    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs\") pod \"e78f1ffe-1463-4430-ae72-919affad054a\" (UID: \"e78f1ffe-1463-4430-ae72-919affad054a\") "
Oct 30 22:20:51.339171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.338884    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls" pod "e78f1ffe-1463-4430-ae72-919affad054a" (UID: "e78f1ffe-1463-4430-ae72-919affad054a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:51.353168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.353137    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs" pod "e78f1ffe-1463-4430-ae72-919affad054a" (UID: "e78f1ffe-1463-4430-ae72-919affad054a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:51.353502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.353479    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca" pod "e78f1ffe-1463-4430-ae72-919affad054a" (UID: "e78f1ffe-1463-4430-ae72-919affad054a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:51.367035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.366595    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5" pod "e78f1ffe-1463-4430-ae72-919affad054a" (UID: "e78f1ffe-1463-4430-ae72-919affad054a"). InnerVolumeSpecName "kube-api-access-9skm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:51.475932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.473581    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9skm5\" (UniqueName: \"kubernetes.io/projected/e78f1ffe-1463-4430-ae72-919affad054a-kube-api-access-9skm5\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:51.475932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.473625    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:51.475932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.473640    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e78f1ffe-1463-4430-ae72-919affad054a-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:51.475932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.473654    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e78f1ffe-1463-4430-ae72-919affad054a-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:51.899382 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.899346    4536 generic.go:356] "Generic (PLEG): container finished" podID="e78f1ffe-1463-4430-ae72-919affad054a" containerID="755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be" exitCode=0
Oct 30 22:20:51.901975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.901951    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"
Oct 30 22:20:51.912468 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.912104    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" event={"ID":"e78f1ffe-1463-4430-ae72-919affad054a","Type":"ContainerDied","Data":"755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be"}
Oct 30 22:20:51.912468 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.912177    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f" event={"ID":"e78f1ffe-1463-4430-ae72-919affad054a","Type":"ContainerDied","Data":"9c491b6533175b0a3ff1a13774d0a77c51507fe1676bbb1d67fc00dd97216008"}
Oct 30 22:20:51.912468 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.912206    4536 scope.go:122] "RemoveContainer" containerID="755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be"
Oct 30 22:20:51.970986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.970959    4536 scope.go:122] "RemoveContainer" containerID="755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be"
Oct 30 22:20:51.971966 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:20:51.971384    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be\": container with ID starting with 755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be not found: ID does not exist" containerID="755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be"
Oct 30 22:20:51.971966 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:51.971465    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be"} err="failed to get container status \"755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be\": rpc error: code = NotFound desc = could not find container \"755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be\": container with ID starting with 755c9744f200d2104dbf9277b6da22fa44f770f1a2d04aa4d9c4a1e7e0ca07be not found: ID does not exist"
Oct 30 22:20:52.093355 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:52.093322    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"]
Oct 30 22:20:52.096509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:52.096489    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea4ac3ec-b7d7bf5b6-rkz9f"]
Oct 30 22:20:52.494561 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:52.494530    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"]
Oct 30 22:20:52.494749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:52.494724    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" podUID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" containerName="maas-api" containerID="cri-o://b77d22653ca72e717ee514325478c0d7192a6e62caa2084c02f0faed6eef4c8a" gracePeriod=30
Oct 30 22:20:52.977221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:52.975660    4536 generic.go:356] "Generic (PLEG): container finished" podID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" containerID="b77d22653ca72e717ee514325478c0d7192a6e62caa2084c02f0faed6eef4c8a" exitCode=0
Oct 30 22:20:52.977221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:52.975772    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" event={"ID":"1f013cb4-6056-4acf-8fb1-10df58a5ce21","Type":"ContainerDied","Data":"b77d22653ca72e717ee514325478c0d7192a6e62caa2084c02f0faed6eef4c8a"}
Oct 30 22:20:53.172274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.170659    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:20:53.243746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.243641    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e78f1ffe-1463-4430-ae72-919affad054a" path="/var/lib/kubelet/pods/e78f1ffe-1463-4430-ae72-919affad054a/volumes"
Oct 30 22:20:53.292653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.292596    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"]
Oct 30 22:20:53.293832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.293804    4536 cpu_manager.go:414] "containerMap: removing container" podUID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" containerName="maas-api"
Oct 30 22:20:53.293984 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.293971    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" containerName="maas-api"
Oct 30 22:20:53.294139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.294128    4536 cpu_manager.go:414] "containerMap: removing container" podUID="7e186f81-bf56-4219-85e8-478b9c77015b" containerName="maas-api"
Oct 30 22:20:53.294225 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.294213    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e186f81-bf56-4219-85e8-478b9c77015b" containerName="maas-api"
Oct 30 22:20:53.294305 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.294296    4536 cpu_manager.go:414] "containerMap: removing container" podUID="b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" containerName="payload-processing"
Oct 30 22:20:53.294376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.294367    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" containerName="payload-processing"
Oct 30 22:20:53.294619 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.294547    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e78f1ffe-1463-4430-ae72-919affad054a" containerName="maas-api"
Oct 30 22:20:53.294800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.294786    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e78f1ffe-1463-4430-ae72-919affad054a" containerName="maas-api"
Oct 30 22:20:53.295222 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.295180    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e186f81-bf56-4219-85e8-478b9c77015b" containerName="maas-api"
Oct 30 22:20:53.295352 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.295339    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3c61af7-fbf8-4ed1-8eaa-55756cd66eec" containerName="payload-processing"
Oct 30 22:20:53.295462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.295451    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e78f1ffe-1463-4430-ae72-919affad054a" containerName="maas-api"
Oct 30 22:20:53.295556 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.295547    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" containerName="maas-api"
Oct 30 22:20:53.317588 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.301835    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca\") pod \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") "
Oct 30 22:20:53.317588 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.301991    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\") pod \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") "
Oct 30 22:20:53.317588 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.302149    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs\") pod \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") "
Oct 30 22:20:53.317588 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.302228    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m\" (UniqueName: \"kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m\") pod \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\" (UID: \"1f013cb4-6056-4acf-8fb1-10df58a5ce21\") "
Oct 30 22:20:53.318837 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.318769    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca" pod "1f013cb4-6056-4acf-8fb1-10df58a5ce21" (UID: "1f013cb4-6056-4acf-8fb1-10df58a5ce21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:53.319929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.319891    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f013cb4-6056-4acf-8fb1-10df58a5ce21-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:53.323163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.323114    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs" pod "1f013cb4-6056-4acf-8fb1-10df58a5ce21" (UID: "1f013cb4-6056-4acf-8fb1-10df58a5ce21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:53.326299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.326233    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m" pod "1f013cb4-6056-4acf-8fb1-10df58a5ce21" (UID: "1f013cb4-6056-4acf-8fb1-10df58a5ce21"). InnerVolumeSpecName "kube-api-access-krb9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:53.331049 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.331014    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"]
Oct 30 22:20:53.331284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.331266    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.337431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.334670    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"]
Oct 30 22:20:53.337431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.334728    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"]
Oct 30 22:20:53.337431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.334794    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.337431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.335517    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls" pod "1f013cb4-6056-4acf-8fb1-10df58a5ce21" (UID: "1f013cb4-6056-4acf-8fb1-10df58a5ce21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:53.420970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.420937    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.420977    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7xrx\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421008    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421071    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421258 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421190    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421263    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421307    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421328    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421453    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421495    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421518    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421578 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421548    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jhhm\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421597    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421663    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421711 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421694    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421715    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421804 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421793    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.421839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421816    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.421937 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421927    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krb9m\" (UniqueName: \"kubernetes.io/projected/1f013cb4-6056-4acf-8fb1-10df58a5ce21-kube-api-access-krb9m\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:53.421976 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421944    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:53.421976 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.421957    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f013cb4-6056-4acf-8fb1-10df58a5ce21-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524507    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524729    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524785    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524819    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jhhm\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524880    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524942    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.524985    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525016    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525074    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525106    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525198    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525227    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7xrx\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525274    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525300    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525345    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.525418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525435    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.526500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525488    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.526500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.525515    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.526500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.526001    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.526500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.526230    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.532182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.526946    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.543467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.538869    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.543467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.540120    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.543467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.541382    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.543467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.542260    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.543467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.543244    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.543870 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.543557    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.543936 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.543870    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.549055 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.549023    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.568415 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.568353    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.571318 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.571288    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.588916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.582950    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.593712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.593684    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jhhm\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.596550 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.596514    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.604043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.604017    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-p5qh2\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.604235 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.604215    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7xrx\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx\") pod \"maas-default-gateway-openshift-default-76597d6855-27rdr\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.664759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.664723    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:53.673836 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.673715    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:53.987543 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.987509    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" event={"ID":"1f013cb4-6056-4acf-8fb1-10df58a5ce21","Type":"ContainerDied","Data":"5227fcd0d8d793997f0a0bb7e77b6095e0ed949c0e40a4a7f399161185fe0191"}
Oct 30 22:20:53.988090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.988062    4536 scope.go:122] "RemoveContainer" containerID="b77d22653ca72e717ee514325478c0d7192a6e62caa2084c02f0faed6eef4c8a"
Oct 30 22:20:53.988251 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:53.988234    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"
Oct 30 22:20:54.159821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.159731    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"]
Oct 30 22:20:54.168029 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.168003    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj"]
Oct 30 22:20:54.298523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.298492    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"]
Oct 30 22:20:54.300259 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:20:54.300235    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f74cfd_0389_41ff_88b5_80e17360fcb8.slice/crio-987031b20abb4be0bd336e25c7ebae78e40edb6e753de5ee74c5d09b247d5366 WatchSource:0}: Error finding container 987031b20abb4be0bd336e25c7ebae78e40edb6e753de5ee74c5d09b247d5366: Status 404 returned error can't find the container with id 987031b20abb4be0bd336e25c7ebae78e40edb6e753de5ee74c5d09b247d5366
Oct 30 22:20:54.303666 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.303631    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:20:54.303858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.303837    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:20:54.303985 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.303969    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:20:54.348634 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:20:54.348604    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd55b555e_0b69_4143_8ed3_95ed8cf39b9d.slice/crio-ae678f7d54a55562db3af4d49592017d475baef0eecf1dab388c8f2c1a4f1654 WatchSource:0}: Error finding container ae678f7d54a55562db3af4d49592017d475baef0eecf1dab388c8f2c1a4f1654: Status 404 returned error can't find the container with id ae678f7d54a55562db3af4d49592017d475baef0eecf1dab388c8f2c1a4f1654
Oct 30 22:20:54.348798 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.348674    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"]
Oct 30 22:20:54.351563 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.351530    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:20:54.351653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.351595    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:20:54.351653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:54.351648    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:20:55.033803 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.031874    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" event={"ID":"40f74cfd-0389-41ff-88b5-80e17360fcb8","Type":"ContainerStarted","Data":"cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716"}
Oct 30 22:20:55.033803 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.031944    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" event={"ID":"40f74cfd-0389-41ff-88b5-80e17360fcb8","Type":"ContainerStarted","Data":"987031b20abb4be0bd336e25c7ebae78e40edb6e753de5ee74c5d09b247d5366"}
Oct 30 22:20:55.056107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.056067    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" event={"ID":"d55b555e-0b69-4143-8ed3-95ed8cf39b9d","Type":"ContainerStarted","Data":"d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf"}
Oct 30 22:20:55.056365 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.056348    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" event={"ID":"d55b555e-0b69-4143-8ed3-95ed8cf39b9d","Type":"ContainerStarted","Data":"ae678f7d54a55562db3af4d49592017d475baef0eecf1dab388c8f2c1a4f1654"}
Oct 30 22:20:55.168298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.168264    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" path="/var/lib/kubelet/pods/1f013cb4-6056-4acf-8fb1-10df58a5ce21/volumes"
Oct 30 22:20:55.170968 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.170929    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podStartSLOduration=2.17091486 podStartE2EDuration="2.17091486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:55.166575987 +0000 UTC m=+1586.154384253" watchObservedRunningTime="2026-10-30 22:20:55.17091486 +0000 UTC m=+1586.158723172"
Oct 30 22:20:55.293940 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.293837    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podStartSLOduration=2.293821878 podStartE2EDuration="2.293821878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:55.290586123 +0000 UTC m=+1586.278394399" watchObservedRunningTime="2026-10-30 22:20:55.293821878 +0000 UTC m=+1586.281630140"
Oct 30 22:20:55.728421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.668678    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:20:55.751673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.751633    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:20:55.752033 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.751994    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:20:55.781645 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.780884    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:20:55.782787 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.782759    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:20:55.783044 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.783002    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:20:55.811428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:55.800043    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4266be31-6444556b-lfmdj" podUID="1f013cb4-6056-4acf-8fb1-10df58a5ce21" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.76:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:20:56.668422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:56.667192    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:20:56.668422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:56.667252    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:20:56.678794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:56.678756    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:20:56.679049 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:56.679019    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:20:56.756502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:56.747234    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"]
Oct 30 22:20:56.756502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:56.747574    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy" containerID="cri-o://d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6" gracePeriod=30
Oct 30 22:20:57.666257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:57.666219    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:20:57.666453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:57.666282    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:20:57.680548 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:57.680504    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:20:57.681138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:57.681108    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:20:58.666264 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:58.666226    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:20:58.666468 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:58.666292    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:20:58.685423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:58.679798    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:20:58.685423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:58.679855    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:20:59.666106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:59.666070    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:20:59.666293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:59.666121    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:20:59.675807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:59.675774    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:20:59.676044 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:20:59.676018    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:21:00.666719 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:00.666685    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:21:00.667130 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:00.666740    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:21:00.676540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:00.676501    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:21:00.693455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:00.693418    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:21:01.666174 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:01.666137    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:21:01.666388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:01.666202    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:21:01.690421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:01.679878    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:21:01.690421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:01.679942    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:21:02.162092 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.162066    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:21:02.250986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.250885    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.250986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.250934    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.250986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.250977    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251033    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251086    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251114    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251185    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251227    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251498 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251310    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5\") pod \"e3619c04-3858-4435-b51a-91b28b1a2265\" (UID: \"e3619c04-3858-4435-b51a-91b28b1a2265\") "
Oct 30 22:21:02.251794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251767    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:02.251875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.251858    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3619c04-3858-4435-b51a-91b28b1a2265-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.252148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.252113    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:02.252365 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.252327    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:02.252365 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.252347    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:02.252646 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.252562    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:02.254095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.254073    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "kube-api-access-sthc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:02.254477 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.254461    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:02.254533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.254464    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:02.254533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.254481    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy" pod "e3619c04-3858-4435-b51a-91b28b1a2265" (UID: "e3619c04-3858-4435-b51a-91b28b1a2265"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:02.343451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.343417    4536 generic.go:356] "Generic (PLEG): container finished" podID="e3619c04-3858-4435-b51a-91b28b1a2265" containerID="d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6" exitCode=0
Oct 30 22:21:02.343648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.343463    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"
Oct 30 22:21:02.343648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.343493    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" event={"ID":"e3619c04-3858-4435-b51a-91b28b1a2265","Type":"ContainerDied","Data":"d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6"}
Oct 30 22:21:02.343648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.343552    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs" event={"ID":"e3619c04-3858-4435-b51a-91b28b1a2265","Type":"ContainerDied","Data":"7e3a2fa66f65f44c9c1925510f48096483bf6ef0287d4337fb437230e9d210df"}
Oct 30 22:21:02.343648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.343580    4536 scope.go:122] "RemoveContainer" containerID="d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6"
Oct 30 22:21:02.354640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354613    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.354732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354646    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.354732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354665    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.354732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354681    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.354732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354696    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.354732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354711    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sthc5\" (UniqueName: \"kubernetes.io/projected/e3619c04-3858-4435-b51a-91b28b1a2265-kube-api-access-sthc5\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.354732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354727    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3619c04-3858-4435-b51a-91b28b1a2265-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.355024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.354744    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3619c04-3858-4435-b51a-91b28b1a2265-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:02.356225 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.356206    4536 scope.go:122] "RemoveContainer" containerID="321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"
Oct 30 22:21:02.370545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.370518    4536 scope.go:122] "RemoveContainer" containerID="d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6"
Oct 30 22:21:02.370995 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:21:02.370971    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6\": container with ID starting with d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6 not found: ID does not exist" containerID="d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6"
Oct 30 22:21:02.371168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.371140    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6"} err="failed to get container status \"d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6\": rpc error: code = NotFound desc = could not find container \"d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6\": container with ID starting with d62a9f6a8e1d557fb54d03e564d18d118876a0528f9dc5da708b1bf15777ccf6 not found: ID does not exist"
Oct 30 22:21:02.371284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.371263    4536 scope.go:122] "RemoveContainer" containerID="321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"
Oct 30 22:21:02.376541 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:21:02.376504    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643\": container with ID starting with 321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643 not found: ID does not exist" containerID="321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"
Oct 30 22:21:02.376722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.376691    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643"} err="failed to get container status \"321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643\": rpc error: code = NotFound desc = could not find container \"321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643\": container with ID starting with 321cc6b0d40aa35ef0aef7b1d75898eb439802096a5a2016370471079c535643 not found: ID does not exist"
Oct 30 22:21:02.481161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.479876    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"]
Oct 30 22:21:02.483653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.483622    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4266be31-openshift-default-5856bd9df5-vqkcs"]
Oct 30 22:21:02.666106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.666015    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:21:02.666106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.666094    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:21:02.674734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.674701    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:21:02.674990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:02.674961    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:21:03.170157 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:03.170120    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" path="/var/lib/kubelet/pods/e3619c04-3858-4435-b51a-91b28b1a2265/volumes"
Oct 30 22:21:03.670425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:03.669898    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-p5qh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:21:03.670425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:03.669971    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:21:03.687438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:03.682017    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:21:03.687438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:03.682091    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:21:04.674933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:04.674897    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-27rdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 22:21:04.675496 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:04.674955    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 22:21:05.508648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:05.508617    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:21:05.509386 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:05.509060    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:21:05.510237 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:05.510218    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:21:05.678273 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:05.678238    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:21:05.678702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:05.678609    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:21:05.679527 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:05.679504    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:21:08.311487 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.311458    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"]
Oct 30 22:21:08.312064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.312050    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy"
Oct 30 22:21:08.312111 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.312065    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy"
Oct 30 22:21:08.312168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.312159    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy"
Oct 30 22:21:08.312211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.312172    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy"
Oct 30 22:21:08.315509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.315493    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396501    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396744 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396569    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pnwb\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396744 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396636    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396744 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396726    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396755    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396786    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396857    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396889    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.396929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.396912    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.414599 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.414569    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"]
Oct 30 22:21:08.498376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498346    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498379    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498602 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498559    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498602 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498585    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pnwb\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498713 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498624    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498713 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498704    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498732    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498767    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498825    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498839    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498861    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.498942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.498926    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.499198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.499175    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.499601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.499576    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.501349 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.501321    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.501653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.501634    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.561372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.561343    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.561541 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.561421    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pnwb\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb\") pod \"maas-default-gateway-openshift-default-76597d6855-qmc8g\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:08.628000 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:08.627966    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:09.065332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.065308    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"]
Oct 30 22:21:09.066633 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:21:09.066609    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4148b30_7fb7_43fe_83da_d0b0169d2284.slice/crio-7a02c33932910205c7ea73f04f81f313d23c3c77d8d793d1241cfd897c27e2bb WatchSource:0}: Error finding container 7a02c33932910205c7ea73f04f81f313d23c3c77d8d793d1241cfd897c27e2bb: Status 404 returned error can't find the container with id 7a02c33932910205c7ea73f04f81f313d23c3c77d8d793d1241cfd897c27e2bb
Oct 30 22:21:09.069171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.069138    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:09.069253 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.069204    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:09.069253 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.069233    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:09.553679 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.553641    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" event={"ID":"c4148b30-7fb7-43fe-83da-d0b0169d2284","Type":"ContainerStarted","Data":"17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770"}
Oct 30 22:21:09.554055 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.553702    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" event={"ID":"c4148b30-7fb7-43fe-83da-d0b0169d2284","Type":"ContainerStarted","Data":"7a02c33932910205c7ea73f04f81f313d23c3c77d8d793d1241cfd897c27e2bb"}
Oct 30 22:21:09.628929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.628897    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:09.679139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:09.679082    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podStartSLOduration=1.6790664629999998 podStartE2EDuration="1.679066463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:09.675852357 +0000 UTC m=+1600.663660619" watchObservedRunningTime="2026-10-30 22:21:09.679066463 +0000 UTC m=+1600.666874726"
Oct 30 22:21:10.629957 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:10.629924    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:10.630361 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:10.629981    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:11.628269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:11.628233    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:11.628465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:11.628284    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:12.629566 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:12.629530    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:12.630241 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:12.630210    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:13.629133 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:13.629100    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:13.629317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:13.629162    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:14.629249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:14.629214    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:14.629755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:14.629284    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:15.629164 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:15.629125    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:15.629343 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:15.629194    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:16.629089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:16.629050    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-qmc8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 22:21:16.629250 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:16.629110    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 22:21:17.752294 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:17.752254    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:17.752795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:17.752654    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:17.753858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:17.753838    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:21:20.806713 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:21:20.806677    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:21:32.679466 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.679432    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"]
Oct 30 22:21:32.680044 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.680031    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy"
Oct 30 22:21:32.680089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.680046    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3619c04-3858-4435-b51a-91b28b1a2265" containerName="istio-proxy"
Oct 30 22:21:32.684946 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.684924    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.739051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.739010    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-05eb684c-serving-cert\""
Oct 30 22:21:32.791547 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.791515    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"]
Oct 30 22:21:32.793565 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.793534    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.793734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.793688    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.793807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.793780    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkjd5\" (UniqueName: \"kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.793864 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.793823    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.895507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.895475    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.895700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.895550    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.895700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.895613    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkjd5\" (UniqueName: \"kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.895700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.895641    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.895853 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:21:32.895708    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-serving-cert: secret "maas-api-e2e-migrate-05eb684c-serving-cert" not found
Oct 30 22:21:32.895853 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:21:32.895782    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls podName:e02a0475-bb14-459a-83b5-967edfc84d09 nodeName:}" failed. No retries permitted until 2026-10-30 22:21:33.395760525 +0000 UTC m=+1624.383568784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls") pod "maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" (UID: "e02a0475-bb14-459a-83b5-967edfc84d09") : secret "maas-api-e2e-migrate-05eb684c-serving-cert" not found
Oct 30 22:21:32.896335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.896315    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.898166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.898141    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:32.983138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:32.983099    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkjd5\" (UniqueName: \"kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:33.406148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:33.406057    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:33.408732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:33.408713    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\") pod \"maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:33.600041 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:33.600004    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:33.987309 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:33.987268    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"]
Oct 30 22:21:33.987883 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:21:33.987855    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02a0475_bb14_459a_83b5_967edfc84d09.slice/crio-e744bb7a733f46cb53ac2c6b3d547efe952b66f65931bd89acbacb8fffec3f83 WatchSource:0}: Error finding container e744bb7a733f46cb53ac2c6b3d547efe952b66f65931bd89acbacb8fffec3f83: Status 404 returned error can't find the container with id e744bb7a733f46cb53ac2c6b3d547efe952b66f65931bd89acbacb8fffec3f83
Oct 30 22:21:34.804367 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:34.804331    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" event={"ID":"e02a0475-bb14-459a-83b5-967edfc84d09","Type":"ContainerStarted","Data":"e744bb7a733f46cb53ac2c6b3d547efe952b66f65931bd89acbacb8fffec3f83"}
Oct 30 22:21:35.340426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.267615    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"]
Oct 30 22:21:35.679479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.673083    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"]
Oct 30 22:21:35.679479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.673220    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.730601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.730570    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.730877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.730857    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792175    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792291    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792372    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knk98\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792433    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792501    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792569    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.792776    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.796503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.794985    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w6-d50b13-openshift-default-dockercfg-7j2cl\""
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.902794    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.902861    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.902920    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.903004    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.903077    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knk98\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.903108    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.903161    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.903215    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.906306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.903413    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.973479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.967477    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.973479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.967846    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:35.973479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.970181    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.049491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:35.998246    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.049491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.011775    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" event={"ID":"e02a0475-bb14-459a-83b5-967edfc84d09","Type":"ContainerStarted","Data":"a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c"}
Oct 30 22:21:36.049491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.012475    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:36.049491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.025295    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.049491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.026985    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.049491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.047662    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.100430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.055687    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.115421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.111146    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knk98\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98\") pod \"e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") " pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:36.156753 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.156680    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" podStartSLOduration=3.312375664 podStartE2EDuration="4.156656703s" totalImagesPullingTime="844.281039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:36.153716303 +0000 UTC m=+1627.141524603" watchObservedRunningTime="2026-10-30 22:21:36.156656703 +0000 UTC m=+1627.144464967"
Oct 30 22:21:36.390094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:36.388701    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:37.558424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:37.556735    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"]
Oct 30 22:21:37.597431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:37.596168    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:37.597431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:37.596334    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:37.597431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:37.596454    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:38.193335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:38.181330    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" event={"ID":"6a8b6174-67fc-4776-bc8f-4a72565638ff","Type":"ContainerStarted","Data":"ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"}
Oct 30 22:21:38.193335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:38.181433    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" event={"ID":"6a8b6174-67fc-4776-bc8f-4a72565638ff","Type":"ContainerStarted","Data":"f1dcf883ffe84194e793c8da95fbd393f0f258a0de02cfff3a2e3d06adb479c7"}
Oct 30 22:21:38.400469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:38.389479    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:38.400469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:38.393463    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podStartSLOduration=3.393443295 podStartE2EDuration="3.393443295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:38.386334928 +0000 UTC m=+1629.374143227" watchObservedRunningTime="2026-10-30 22:21:38.393443295 +0000 UTC m=+1629.381251559"
Oct 30 22:21:39.226440 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:39.218265    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:21:39.423172 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:39.423138    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:40.219776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:40.219747    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:40.220762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:40.220743    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:21:42.142958 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:42.142799    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:21:45.043240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.043201    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"]
Oct 30 22:21:45.078289 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.070712    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.116221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.109061    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"]
Oct 30 22:21:45.233674 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.233642    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w6-d50b13\""
Oct 30 22:21:45.250572 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.233961    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w6-d50b13-dockercfg-clmc2\""
Oct 30 22:21:45.306306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.274947    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-686xw\" (UniqueName: \"kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw\") pod \"payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.331469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.327315    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.331469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.299219    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"]
Oct 30 22:21:45.462465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.448169    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-686xw\" (UniqueName: \"kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw\") pod \"payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.462465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.448325    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.462465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.454214    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.480879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.480578    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"]
Oct 30 22:21:45.484701 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.484675    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.558480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.550409    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.558480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.551060    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xsgh\" (UniqueName: \"kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh\") pod \"payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.574916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.574821    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-686xw\" (UniqueName: \"kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw\") pod \"payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.653066 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.653029    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xsgh\" (UniqueName: \"kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh\") pod \"payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.653259 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.653123    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.654000 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.653979    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.715749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.715715    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xsgh\" (UniqueName: \"kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh\") pod \"payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:45.790559 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.790527    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:45.833849 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:45.833754    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:46.171168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.171143    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"]
Oct 30 22:21:46.171738 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:21:46.171704    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657220e1_3d93_4875_8a4c_b727c3e9751e.slice/crio-be66c5b3440402eb853aef749368ea7ac20adfcf929724459f6f5125bcd59264 WatchSource:0}: Error finding container be66c5b3440402eb853aef749368ea7ac20adfcf929724459f6f5125bcd59264: Status 404 returned error can't find the container with id be66c5b3440402eb853aef749368ea7ac20adfcf929724459f6f5125bcd59264
Oct 30 22:21:46.235242 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.235218    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"]
Oct 30 22:21:46.236146 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:21:46.236119    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde107361_6bf7_462d_abd8_3ef49de8e097.slice/crio-b7080ea4ff7c0ed53cfb9639fbd9cafde83be3947cda903c9bcaaea30a1d9b0a WatchSource:0}: Error finding container b7080ea4ff7c0ed53cfb9639fbd9cafde83be3947cda903c9bcaaea30a1d9b0a: Status 404 returned error can't find the container with id b7080ea4ff7c0ed53cfb9639fbd9cafde83be3947cda903c9bcaaea30a1d9b0a
Oct 30 22:21:46.503496 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.503447    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn" event={"ID":"de107361-6bf7-462d-abd8-3ef49de8e097","Type":"ContainerStarted","Data":"8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967"}
Oct 30 22:21:46.503716 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.503510    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn" event={"ID":"de107361-6bf7-462d-abd8-3ef49de8e097","Type":"ContainerStarted","Data":"b7080ea4ff7c0ed53cfb9639fbd9cafde83be3947cda903c9bcaaea30a1d9b0a"}
Oct 30 22:21:46.503716 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.503539    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:46.505121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.505086    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742" event={"ID":"657220e1-3d93-4875-8a4c-b727c3e9751e","Type":"ContainerStarted","Data":"aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1"}
Oct 30 22:21:46.505234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.505129    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742" event={"ID":"657220e1-3d93-4875-8a4c-b727c3e9751e","Type":"ContainerStarted","Data":"be66c5b3440402eb853aef749368ea7ac20adfcf929724459f6f5125bcd59264"}
Oct 30 22:21:46.505234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.505151    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:46.623490 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:46.623430    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn" podStartSLOduration=1.623389086 podStartE2EDuration="1.623389086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:46.621789085 +0000 UTC m=+1637.609597348" watchObservedRunningTime="2026-10-30 22:21:46.623389086 +0000 UTC m=+1637.611197407"
Oct 30 22:21:49.741681 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.741631    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742" podStartSLOduration=5.741616504 podStartE2EDuration="5.741616504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:46.757278925 +0000 UTC m=+1637.745087204" watchObservedRunningTime="2026-10-30 22:21:49.741616504 +0000 UTC m=+1640.729424767"
Oct 30 22:21:49.742078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.741912    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"]
Oct 30 22:21:49.746017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.745999    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.801133 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.801100    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-2dd57a-openshift-default-dockercfg-xqhgr\""
Oct 30 22:21:49.845152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.844789    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.844907    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5p22\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845049    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845074    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845095    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845135    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845226    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845336    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.845516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.845438    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.855371 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.855340    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"]
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.949629    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.949727    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.949814    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5p22\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.949951    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.949982    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.950011    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.950055    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.950150    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.950192    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.968846    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.974426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.969127    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:49.993469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:49.987206    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.051505 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.017347    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.121433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.116581    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.121433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.117935    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5p22\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.121965 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.121941    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.218420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.178871    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.218420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.184096    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy\") pod \"e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") " pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.374803 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:50.374717    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:50.936510 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:21:50.932017    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:21:51.362573 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.362544    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"]
Oct 30 22:21:51.363703 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:21:51.363673    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe027fb4_c600_4b5d_804e_966790379969.slice/crio-0f1d2a5a0267267f538fe8f8052663d5b02a43381ea800825ce5c4440df9620a WatchSource:0}: Error finding container 0f1d2a5a0267267f538fe8f8052663d5b02a43381ea800825ce5c4440df9620a: Status 404 returned error can't find the container with id 0f1d2a5a0267267f538fe8f8052663d5b02a43381ea800825ce5c4440df9620a
Oct 30 22:21:51.367103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.367063    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:51.367191 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.367150    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:51.367227 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.367209    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:21:51.634117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.634013    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" event={"ID":"fe027fb4-c600-4b5d-804e-966790379969","Type":"ContainerStarted","Data":"c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"}
Oct 30 22:21:51.634117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.634081    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" event={"ID":"fe027fb4-c600-4b5d-804e-966790379969","Type":"ContainerStarted","Data":"0f1d2a5a0267267f538fe8f8052663d5b02a43381ea800825ce5c4440df9620a"}
Oct 30 22:21:51.770329 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:51.770264    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podStartSLOduration=2.770245491 podStartE2EDuration="2.770245491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:51.767736722 +0000 UTC m=+1642.755544988" watchObservedRunningTime="2026-10-30 22:21:51.770245491 +0000 UTC m=+1642.758053755"
Oct 30 22:21:52.377555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:52.377526    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:52.383510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:52.383487    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:52.663481 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:52.651435    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:52.663481 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:52.658716    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:21:57.517411 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:57.517366    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:21:57.517884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:57.517477    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:21:59.810105 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.810075    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"]
Oct 30 22:21:59.814618 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.814602    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:21:59.873319 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.873284    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"]
Oct 30 22:21:59.902349 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.902210    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tkbm\" (UniqueName: \"kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm\") pod \"payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:21:59.902742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.902387    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:21:59.953421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.939290    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2dd57a\""
Oct 30 22:21:59.953421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:21:59.939478    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2dd57a-dockercfg-svk9w\""
Oct 30 22:22:00.017007 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.010668    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:00.017007 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.010978    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tkbm\" (UniqueName: \"kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm\") pod \"payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:00.017007 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.012226    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:00.217091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.206076    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tkbm\" (UniqueName: \"kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm\") pod \"payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:00.300315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.225613    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"]
Oct 30 22:22:00.306175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.305066    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.311040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.311006    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thn79\" (UniqueName: \"kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79\") pod \"payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.311176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.311143    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.337969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.336212    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"]
Oct 30 22:22:00.412642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.412608    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thn79\" (UniqueName: \"kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79\") pod \"payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.412832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.412694    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.413654 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.413631    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.430292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.430251    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:00.476017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.475950    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thn79\" (UniqueName: \"kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79\") pod \"payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.628386 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.628352    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:00.829260 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:00.829230    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"]
Oct 30 22:22:00.830512 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:22:00.830468    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe96df3d_6aa7_4826_bacb_bf548e0beca8.slice/crio-41b8749ae29d5998b9e075199955e282e38134a78d520a88ab2adeb689496306 WatchSource:0}: Error finding container 41b8749ae29d5998b9e075199955e282e38134a78d520a88ab2adeb689496306: Status 404 returned error can't find the container with id 41b8749ae29d5998b9e075199955e282e38134a78d520a88ab2adeb689496306
Oct 30 22:22:01.015520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.015481    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"]
Oct 30 22:22:01.015787 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:22:01.015765    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc894bbd_6a33_47e0_a429_0e88deb98485.slice/crio-69068a2f406bd827649d6537b481ba77fbd0a5a5c0da868db071de4dc2b0068d WatchSource:0}: Error finding container 69068a2f406bd827649d6537b481ba77fbd0a5a5c0da868db071de4dc2b0068d: Status 404 returned error can't find the container with id 69068a2f406bd827649d6537b481ba77fbd0a5a5c0da868db071de4dc2b0068d
Oct 30 22:22:01.781562 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.781523    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v" event={"ID":"be96df3d-6aa7-4826-bacb-bf548e0beca8","Type":"ContainerStarted","Data":"4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd"}
Oct 30 22:22:01.781767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.781576    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v" event={"ID":"be96df3d-6aa7-4826-bacb-bf548e0beca8","Type":"ContainerStarted","Data":"41b8749ae29d5998b9e075199955e282e38134a78d520a88ab2adeb689496306"}
Oct 30 22:22:01.781767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.781730    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:01.783340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.783315    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh" event={"ID":"fc894bbd-6a33-47e0-a429-0e88deb98485","Type":"ContainerStarted","Data":"761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6"}
Oct 30 22:22:01.783484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.783358    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh" event={"ID":"fc894bbd-6a33-47e0-a429-0e88deb98485","Type":"ContainerStarted","Data":"69068a2f406bd827649d6537b481ba77fbd0a5a5c0da868db071de4dc2b0068d"}
Oct 30 22:22:01.783484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.783466    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:01.903247 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:01.903193    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v" podStartSLOduration=2.9031797409999998 podStartE2EDuration="2.903179741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:01.900667588 +0000 UTC m=+1652.888475853" watchObservedRunningTime="2026-10-30 22:22:01.903179741 +0000 UTC m=+1652.890988031"
Oct 30 22:22:02.064473 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:02.058034    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh" podStartSLOduration=3.058008448 podStartE2EDuration="3.058008448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:02.033841468 +0000 UTC m=+1653.021649732" watchObservedRunningTime="2026-10-30 22:22:02.058008448 +0000 UTC m=+1653.045816712"
Oct 30 22:22:12.792929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:12.792854    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:22:12.792929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:12.792929    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:22:20.808271 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:22:20.808231    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:22:36.942049 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:36.942013    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"]
Oct 30 22:22:36.946494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:36.946475    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:36.968116 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:36.968089    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:36.968266 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:36.968128    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:36.968266 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:36.968262    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8rxl\" (UniqueName: \"kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:36.968344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:36.968301    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.000730 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.000703    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-d50b13-serving-cert\""
Oct 30 22:22:37.053612 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.053584    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"]
Oct 30 22:22:37.070844 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.070810    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.071039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.070861    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.071039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.070987    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8rxl\" (UniqueName: \"kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.071174 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.071048    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.071245 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:22:37.071195    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-serving-cert: secret "maas-api-e2e-shared-a-w6-d50b13-serving-cert" not found
Oct 30 22:22:37.071309 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:22:37.071255    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls podName:e3ded310-d258-4da0-81e5-dc7924ca1886 nodeName:}" failed. No retries permitted until 2026-10-30 22:22:37.571236053 +0000 UTC m=+1688.559044308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls") pod "maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" (UID: "e3ded310-d258-4da0-81e5-dc7924ca1886") : secret "maas-api-e2e-shared-a-w6-d50b13-serving-cert" not found
Oct 30 22:22:37.072961 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.072933    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.076842 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.076819    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.135366 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.135325    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8rxl\" (UniqueName: \"kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.609420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.605660    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.622447 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.621781    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:37.860384 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:37.860293    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:38.248717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:38.248687    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"]
Oct 30 22:22:38.249759 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:22:38.249734    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ded310_d258_4da0_81e5_dc7924ca1886.slice/crio-0bd7582599089ecf3e36e72fe6da14c70376146911f634ff981369be7d62becd WatchSource:0}: Error finding container 0bd7582599089ecf3e36e72fe6da14c70376146911f634ff981369be7d62becd: Status 404 returned error can't find the container with id 0bd7582599089ecf3e36e72fe6da14c70376146911f634ff981369be7d62becd
Oct 30 22:22:39.086580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:39.086546    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" event={"ID":"e3ded310-d258-4da0-81e5-dc7924ca1886","Type":"ContainerStarted","Data":"0bd7582599089ecf3e36e72fe6da14c70376146911f634ff981369be7d62becd"}
Oct 30 22:22:40.094112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:40.094073    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" event={"ID":"e3ded310-d258-4da0-81e5-dc7924ca1886","Type":"ContainerStarted","Data":"1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5"}
Oct 30 22:22:40.094567 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:40.094167    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:40.219290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:40.219237    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" podStartSLOduration=3.420697648 podStartE2EDuration="4.219223531s" totalImagesPullingTime="798.525883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:40.215846753 +0000 UTC m=+1691.203655016" watchObservedRunningTime="2026-10-30 22:22:40.219223531 +0000 UTC m=+1691.207031793"
Oct 30 22:22:46.106916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:22:46.106886    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:22:50.807458 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:22:50.807422    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:23:20.806203 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:20.806166    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:23:32.834288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:32.834243    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:23:32.835230 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:32.834914    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:23:36.321191 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.321156    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"]
Oct 30 22:23:36.321662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.321363    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" podUID="e02a0475-bb14-459a-83b5-967edfc84d09" containerName="maas-api" containerID="cri-o://a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c" gracePeriod=30
Oct 30 22:23:36.888422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.882445    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:23:36.958990 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.958957    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs\") pod \"e02a0475-bb14-459a-83b5-967edfc84d09\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") "
Oct 30 22:23:36.959527 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.959504    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\") pod \"e02a0475-bb14-459a-83b5-967edfc84d09\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") "
Oct 30 22:23:36.959808 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.959776    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca\") pod \"e02a0475-bb14-459a-83b5-967edfc84d09\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") "
Oct 30 22:23:36.960045 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.960015    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5\" (UniqueName: \"kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5\") pod \"e02a0475-bb14-459a-83b5-967edfc84d09\" (UID: \"e02a0475-bb14-459a-83b5-967edfc84d09\") "
Oct 30 22:23:36.974662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.968373    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca" pod "e02a0475-bb14-459a-83b5-967edfc84d09" (UID: "e02a0475-bb14-459a-83b5-967edfc84d09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:36.978420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.975189    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs" pod "e02a0475-bb14-459a-83b5-967edfc84d09" (UID: "e02a0475-bb14-459a-83b5-967edfc84d09"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:23:36.985927 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:36.985890    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5" pod "e02a0475-bb14-459a-83b5-967edfc84d09" (UID: "e02a0475-bb14-459a-83b5-967edfc84d09"). InnerVolumeSpecName "kube-api-access-vkjd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:37.002193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.002125    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls" pod "e02a0475-bb14-459a-83b5-967edfc84d09" (UID: "e02a0475-bb14-459a-83b5-967edfc84d09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:23:37.065502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.064713    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:37.065502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.064747    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e02a0475-bb14-459a-83b5-967edfc84d09-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:37.065502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.064761    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkjd5\" (UniqueName: \"kubernetes.io/projected/e02a0475-bb14-459a-83b5-967edfc84d09-kube-api-access-vkjd5\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:37.065502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.064776    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e02a0475-bb14-459a-83b5-967edfc84d09-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:37.079452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.076159    4536 generic.go:356] "Generic (PLEG): container finished" podID="e02a0475-bb14-459a-83b5-967edfc84d09" containerID="a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c" exitCode=0
Oct 30 22:23:37.079452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.076374    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" event={"ID":"e02a0475-bb14-459a-83b5-967edfc84d09","Type":"ContainerDied","Data":"a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c"}
Oct 30 22:23:37.079452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.076451    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z" event={"ID":"e02a0475-bb14-459a-83b5-967edfc84d09","Type":"ContainerDied","Data":"e744bb7a733f46cb53ac2c6b3d547efe952b66f65931bd89acbacb8fffec3f83"}
Oct 30 22:23:37.079452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.076480    4536 scope.go:122] "RemoveContainer" containerID="a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c"
Oct 30 22:23:37.079452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.076671    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"
Oct 30 22:23:37.120272 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.120227    4536 scope.go:122] "RemoveContainer" containerID="a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c"
Oct 30 22:23:37.120713 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:37.120685    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c\": container with ID starting with a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c not found: ID does not exist" containerID="a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c"
Oct 30 22:23:37.120844 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.120717    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c"} err="failed to get container status \"a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c\": rpc error: code = NotFound desc = could not find container \"a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c\": container with ID starting with a31ccd691d04a11f6c7f85368afad8a5decc9ba5b9e3034374676b7980528a3c not found: ID does not exist"
Oct 30 22:23:37.283662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.283622    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"]
Oct 30 22:23:37.288851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.288827    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-05eb684c-7fd97b49d9-xmf7z"]
Oct 30 22:23:37.431105 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.431027    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"]
Oct 30 22:23:37.431540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.431251    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy" containerID="cri-o://ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f" gracePeriod=30
Oct 30 22:23:37.434668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.434632    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"]
Oct 30 22:23:37.434895 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.434869    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy" containerID="cri-o://433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57" gracePeriod=30
Oct 30 22:23:37.437791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.437763    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"]
Oct 30 22:23:37.437960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:37.437941    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy" containerID="cri-o://9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798" gracePeriod=30
Oct 30 22:23:39.226421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:39.224530    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e02a0475-bb14-459a-83b5-967edfc84d09" path="/var/lib/kubelet/pods/e02a0475-bb14-459a-83b5-967edfc84d09/volumes"
Oct 30 22:23:42.738646 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.738612    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:23:42.781087 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.780966    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:23:42.788626 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.788604    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:23:42.855930 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.855896    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.855950    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856037    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856073    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856092    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856333 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856231    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856333 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856313    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856333 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856299    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.856534 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856371    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.856534 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856386    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.856647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856570    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.856647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856595    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.856647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856617    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:42.856867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.856648    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k\") pod \"63c901e3-b33d-4573-b99f-42929b91502e\" (UID: \"63c901e3-b33d-4573-b99f-42929b91502e\") "
Oct 30 22:23:42.857348 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.857301    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.857348 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.857323    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.857348 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.857332    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.857348 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.857342    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c901e3-b33d-4573-b99f-42929b91502e-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.857659 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.857353    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.858722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.858701    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:42.858809 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.858741    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "kube-api-access-lpr4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:42.858888 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.858875    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:42.858925 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.858884    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy" pod "63c901e3-b33d-4573-b99f-42929b91502e" (UID: "63c901e3-b33d-4573-b99f-42929b91502e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.958523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958435    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.958523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958485    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.958778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958576    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.958778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958661    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.958778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958707    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.958778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958751    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.959004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958783    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958810    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958872    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958873    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.959004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.958984    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959011    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959077    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959117    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959156    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959180    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959212    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket\") pod \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\" (UID: \"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0\") "
Oct 30 22:23:42.959255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959249    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.959634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959321    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.959634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959360    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket\") pod \"adb92766-aef3-4e47-bd55-0040f9090661\" (UID: \"adb92766-aef3-4e47-bd55-0040f9090661\") "
Oct 30 22:23:42.959634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959348    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.959634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959453    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.959634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959591    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.959892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959792    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.959892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.959823    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:42.960249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960223    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960256    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960271    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960285    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960299    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c901e3-b33d-4573-b99f-42929b91502e-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960311    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960325    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c901e3-b33d-4573-b99f-42929b91502e-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960338    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpr4k\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-kube-api-access-lpr4k\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960354    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960367    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c901e3-b33d-4573-b99f-42929b91502e-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960383    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:42.960830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960610    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.960830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.960810    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.961107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.961082    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:42.963323 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.963298    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.963717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.963697    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "kube-api-access-5cnw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.969629    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"]
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970574    4536 cpu_manager.go:414] "containerMap: removing container" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970593    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970627    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970635    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970673    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e02a0475-bb14-459a-83b5-967edfc84d09" containerName="maas-api"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970680    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e02a0475-bb14-459a-83b5-967edfc84d09" containerName="maas-api"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970691    4536 cpu_manager.go:414] "containerMap: removing container" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970698    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970879    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970893    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="63c901e3-b33d-4573-b99f-42929b91502e" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970908    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="adb92766-aef3-4e47-bd55-0040f9090661" containerName="istio-proxy"
Oct 30 22:23:42.971298 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.970918    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e02a0475-bb14-459a-83b5-967edfc84d09" containerName="maas-api"
Oct 30 22:23:42.973003 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.972978    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:42.973111 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.972998    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:42.973111 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.973038    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "kube-api-access-6dg4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:42.973189 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.973116    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:42.973317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.973297    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo" pod "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" (UID: "e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:42.974564 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.974543    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo" pod "adb92766-aef3-4e47-bd55-0040f9090661" (UID: "adb92766-aef3-4e47-bd55-0040f9090661"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:42.976560 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:42.976539    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.012136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.012109    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"]
Oct 30 22:23:43.071776 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.071745    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-367f4491-openshift-default-dockercfg-d7tpf\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079014    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079074    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079158    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079227    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079386    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079439    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079489    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcghq\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079526    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079613    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079706    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cnw9\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-kube-api-access-5cnw9\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079722    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079736    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dg4v\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-kube-api-access-6dg4v\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079750    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079763    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079776    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adb92766-aef3-4e47-bd55-0040f9090661-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079789    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adb92766-aef3-4e47-bd55-0040f9090661-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079803    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adb92766-aef3-4e47-bd55-0040f9090661-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079816    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079830    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adb92766-aef3-4e47-bd55-0040f9090661-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.080182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.079843    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:43.279960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.279862    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.279960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.279919    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280006    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280079    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280241    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280272    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280440 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280322    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcghq\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280440 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280359    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.280558 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.280465    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.281216 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.281191    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.283790 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.281741    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.283790 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.283542    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.283790 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.283658    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.284039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.283864    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.284341 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.284296    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.289657 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.289627    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.316815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.316782    4536 generic.go:356] "Generic (PLEG): container finished" podID="63c901e3-b33d-4573-b99f-42929b91502e" containerID="ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f" exitCode=0
Oct 30 22:23:43.317090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.317065    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" event={"ID":"63c901e3-b33d-4573-b99f-42929b91502e","Type":"ContainerDied","Data":"ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f"}
Oct 30 22:23:43.317265 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.317248    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj" event={"ID":"63c901e3-b33d-4573-b99f-42929b91502e","Type":"ContainerDied","Data":"d81e7091e21b9c3a6c57c8849f205ac276d98c9bff21cc7b3d00c1e7018651c4"}
Oct 30 22:23:43.317412 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.317378    4536 scope.go:122] "RemoveContainer" containerID="ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f"
Oct 30 22:23:43.317534 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.317512    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"
Oct 30 22:23:43.343712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.343643    4536 generic.go:356] "Generic (PLEG): container finished" podID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" containerID="9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798" exitCode=0
Oct 30 22:23:43.346063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.345483    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"
Oct 30 22:23:43.346063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.345504    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" event={"ID":"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0","Type":"ContainerDied","Data":"9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798"}
Oct 30 22:23:43.346063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.345572    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h" event={"ID":"e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0","Type":"ContainerDied","Data":"88607ee33a09d80db1f5c060abafac08fce3270f160aa8007489fb4e7d2fd375"}
Oct 30 22:23:43.349642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.349481    4536 generic.go:356] "Generic (PLEG): container finished" podID="adb92766-aef3-4e47-bd55-0040f9090661" containerID="433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57" exitCode=0
Oct 30 22:23:43.349642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.349532    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" event={"ID":"adb92766-aef3-4e47-bd55-0040f9090661","Type":"ContainerDied","Data":"433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57"}
Oct 30 22:23:43.349642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.349578    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457" event={"ID":"adb92766-aef3-4e47-bd55-0040f9090661","Type":"ContainerDied","Data":"45dc654622d86d82fece24950b48fe59e67f5e7c1d87cb889ef2b3c47ee95265"}
Oct 30 22:23:43.349896 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.349876    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"
Oct 30 22:23:43.352119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.352099    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.354688 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.354665    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcghq\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.362591 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.362558    4536 scope.go:122] "RemoveContainer" containerID="ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f"
Oct 30 22:23:43.362998 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:43.362972    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f\": container with ID starting with ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f not found: ID does not exist" containerID="ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f"
Oct 30 22:23:43.363182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.363155    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f"} err="failed to get container status \"ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f\": rpc error: code = NotFound desc = could not find container \"ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f\": container with ID starting with ace15e11103d3fd25b92a15568543f885609f5031a3c67940ddff0e248fb063f not found: ID does not exist"
Oct 30 22:23:43.363314 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.363301    4536 scope.go:122] "RemoveContainer" containerID="9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798"
Oct 30 22:23:43.377595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.377574    4536 scope.go:122] "RemoveContainer" containerID="9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798"
Oct 30 22:23:43.378016 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:43.377983    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798\": container with ID starting with 9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798 not found: ID does not exist" containerID="9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798"
Oct 30 22:23:43.378124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.378014    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798"} err="failed to get container status \"9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798\": rpc error: code = NotFound desc = could not find container \"9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798\": container with ID starting with 9fe0a50cbe9d74528ec75e37c106cd3369e276f37439c00df9ef2321127b9798 not found: ID does not exist"
Oct 30 22:23:43.378124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.378037    4536 scope.go:122] "RemoveContainer" containerID="433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57"
Oct 30 22:23:43.390342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.390322    4536 scope.go:122] "RemoveContainer" containerID="433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57"
Oct 30 22:23:43.390707 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:43.390687    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57\": container with ID starting with 433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57 not found: ID does not exist" containerID="433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57"
Oct 30 22:23:43.390789 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.390713    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57"} err="failed to get container status \"433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57\": rpc error: code = NotFound desc = could not find container \"433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57\": container with ID starting with 433aa9a82ab9329db6f33ceace6630ba27e2b84daa615fe31f026d832d529e57 not found: ID does not exist"
Oct 30 22:23:43.508524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.508486    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"]
Oct 30 22:23:43.519011 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.518984    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-vb457"]
Oct 30 22:23:43.590979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.590887    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:43.629097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.629062    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"]
Oct 30 22:23:43.639695 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.639669    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-d9j7h"]
Oct 30 22:23:43.751557 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.751528    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"]
Oct 30 22:23:43.757717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:43.757692    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-lrbcj"]
Oct 30 22:23:44.028918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.028885    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"]
Oct 30 22:23:44.029274 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:23:44.029248    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30cbd0cb_0e32_41e1_ba9d_013b5f3c2c3c.slice/crio-8edbb2d69f79d88cb3635bd700c67f365991ccd6673baf8c23d59ce5802d11db WatchSource:0}: Error finding container 8edbb2d69f79d88cb3635bd700c67f365991ccd6673baf8c23d59ce5802d11db: Status 404 returned error can't find the container with id 8edbb2d69f79d88cb3635bd700c67f365991ccd6673baf8c23d59ce5802d11db
Oct 30 22:23:44.032100 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.032067    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:23:44.032209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.032142    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:23:44.032209 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.032192    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:23:44.370831 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.370741    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" event={"ID":"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c","Type":"ContainerStarted","Data":"27bcde5ab3623921fc7dce39f009764e5b6eda64e003000a25d62d915cc017c5"}
Oct 30 22:23:44.371013 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.370830    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" event={"ID":"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c","Type":"ContainerStarted","Data":"8edbb2d69f79d88cb3635bd700c67f365991ccd6673baf8c23d59ce5802d11db"}
Oct 30 22:23:44.495554 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.495501    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podStartSLOduration=2.49548555 podStartE2EDuration="2.49548555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:44.493767818 +0000 UTC m=+1755.481576093" watchObservedRunningTime="2026-10-30 22:23:44.49548555 +0000 UTC m=+1755.483293813"
Oct 30 22:23:44.592029 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:44.591987    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:45.204647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:45.204613    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63c901e3-b33d-4573-b99f-42929b91502e" path="/var/lib/kubelet/pods/63c901e3-b33d-4573-b99f-42929b91502e/volumes"
Oct 30 22:23:45.205335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:45.205314    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="adb92766-aef3-4e47-bd55-0040f9090661" path="/var/lib/kubelet/pods/adb92766-aef3-4e47-bd55-0040f9090661/volumes"
Oct 30 22:23:45.205843 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:45.205825    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0" path="/var/lib/kubelet/pods/e7ee2b3b-0d0b-4c7c-8bff-d0b50f72f6e0/volumes"
Oct 30 22:23:45.602484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:45.595517    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:23:45.602484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:45.595586    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:23:46.593873 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:46.593834    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:23:46.594501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:46.594468    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:23:47.591838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:47.591800    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:23:47.592107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:47.592078    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:23:48.591815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:48.591777    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:23:48.592510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:48.592478    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:23:49.738324 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:49.738283    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:23:49.739205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:49.739170    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:23:50.651429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:50.648907    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:23:50.651429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:50.648966    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:23:50.846113 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:50.846080    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:23:51.601183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:51.601147    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:51.601452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:51.601429    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:51.602106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:51.602089    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:23:52.434952 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:52.434919    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"]
Oct 30 22:23:52.435501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:52.435120    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy" containerID="cri-o://976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3" gracePeriod=30
Oct 30 22:23:52.857225 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:52.857196    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"]
Oct 30 22:23:52.857453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:52.857429    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" containerID="cri-o://bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7" gracePeriod=30
Oct 30 22:23:53.540961 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:53.540920    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"]
Oct 30 22:23:53.541499 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:53.541174    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy" containerID="cri-o://4dc8eb11653e76e7b6ed2f82b565f67b47320498b0c3af05f729466a63242501" gracePeriod=30
Oct 30 22:23:56.030569 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.030541    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"]
Oct 30 22:23:56.035557 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.035536    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.094416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.094370    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-6c7d3f-openshift-default-dockercfg-7zkb2\""
Oct 30 22:23:56.095318 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095293    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg5nl\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095427 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095377    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095478 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095437    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095527 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095511    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095544    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095612    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095676    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095763 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095727    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.095763 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.095775    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.150320 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.150289    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"]
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.221932    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222024    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222081    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222167    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg5nl\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222254    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222299    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222419    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222458    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.241462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.222538    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.242327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.241800    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.256335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.256301    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.273666 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.257092    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.274156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.265480    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.274761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.274720    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.288945 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.285877    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.358190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.358005    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.359826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.359744    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.380285 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.380218    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg5nl\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl\") pod \"e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") " pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:56.651423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:56.651303    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:57.093513 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:23:57.093470    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaef18ca_2ab9_4f4d_9611_7e7b49314f18.slice/crio-72247176b9d3149cfa768097d0dbf44a69953567cbda4317686f93d781d534ac WatchSource:0}: Error finding container 72247176b9d3149cfa768097d0dbf44a69953567cbda4317686f93d781d534ac: Status 404 returned error can't find the container with id 72247176b9d3149cfa768097d0dbf44a69953567cbda4317686f93d781d534ac
Oct 30 22:23:57.094090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.094018    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"]
Oct 30 22:23:57.096156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.096122    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:23:57.096240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.096189    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:23:57.096240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.096218    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:23:57.446197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.446156    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:23:57.446373 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.446227    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:57.736415 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.736375    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:23:57.761064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.761025    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" event={"ID":"faef18ca-2ab9-4f4d-9611-7e7b49314f18","Type":"ContainerStarted","Data":"26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"}
Oct 30 22:23:57.761245 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.761108    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" event={"ID":"faef18ca-2ab9-4f4d-9611-7e7b49314f18","Type":"ContainerStarted","Data":"72247176b9d3149cfa768097d0dbf44a69953567cbda4317686f93d781d534ac"}
Oct 30 22:23:57.762648 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.762616    4536 generic.go:356] "Generic (PLEG): container finished" podID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerID="976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3" exitCode=0
Oct 30 22:23:57.762791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.762650    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"
Oct 30 22:23:57.762791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.762706    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" event={"ID":"a5c61b4c-c214-4eb0-acbe-6515a319d684","Type":"ContainerDied","Data":"976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3"}
Oct 30 22:23:57.762791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.762747    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt" event={"ID":"a5c61b4c-c214-4eb0-acbe-6515a319d684","Type":"ContainerDied","Data":"c85cbb0e75734f63d927698a1e8546b2a0e4c794ad84d85fb005259faac1b1d0"}
Oct 30 22:23:57.762791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.762779    4536 scope.go:122] "RemoveContainer" containerID="976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3"
Oct 30 22:23:57.773695 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.773674    4536 scope.go:122] "RemoveContainer" containerID="976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3"
Oct 30 22:23:57.774013 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:57.773993    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3\": container with ID starting with 976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3 not found: ID does not exist" containerID="976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3"
Oct 30 22:23:57.774066 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.774023    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3"} err="failed to get container status \"976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3\": rpc error: code = NotFound desc = could not find container \"976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3\": container with ID starting with 976ed7cd6117bb01009515d72969deccc4337e3f1d27f5a8f992995d890238d3 not found: ID does not exist"
Oct 30 22:23:57.803131 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803101    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803144    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803169    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803260    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803324    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803435    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803459    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:57.803580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803481    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803630    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803768    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:57.803832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803771    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2\") pod \"a5c61b4c-c214-4eb0-acbe-6515a319d684\" (UID: \"a5c61b4c-c214-4eb0-acbe-6515a319d684\") "
Oct 30 22:23:57.803832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803789    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:57.804237 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.803981    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:57.804237 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.804063    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:57.805149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.805123    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.805240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.805153    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.805240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.805189    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.805240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.805206    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5c61b4c-c214-4eb0-acbe-6515a319d684-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.805240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.805220    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.806220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.806197    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:57.806530 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.806501    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:57.806650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.806545    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:57.806916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.806890    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2" pod "a5c61b4c-c214-4eb0-acbe-6515a319d684" (UID: "a5c61b4c-c214-4eb0-acbe-6515a319d684"). InnerVolumeSpecName "kube-api-access-ckrj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:57.907556 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.907514    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.907556 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.907559    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckrj2\" (UniqueName: \"kubernetes.io/projected/a5c61b4c-c214-4eb0-acbe-6515a319d684-kube-api-access-ckrj2\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.907816 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.907575    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:57.907816 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:57.907589    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5c61b4c-c214-4eb0-acbe-6515a319d684-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.049590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.047538    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podStartSLOduration=3.04751978 podStartE2EDuration="3.04751978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:58.046694107 +0000 UTC m=+1769.034502363" watchObservedRunningTime="2026-10-30 22:23:58.04751978 +0000 UTC m=+1769.035328045"
Oct 30 22:23:58.134051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.134026    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:23:58.213936 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.213842    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"]
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.240573    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.240736    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.240828    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.240983    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.241017    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.241058    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.241103    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.241131    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.241237    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh\") pod \"9ac46abd-764c-465d-9952-af513f3af401\" (UID: \"9ac46abd-764c-465d-9952-af513f3af401\") "
Oct 30 22:23:58.242538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.242270    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.243175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.242567    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.243175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.242735    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.243646 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.243348    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.247292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.245765    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:58.247292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.246112    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:58.248576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.248553    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-wz6rt"]
Oct 30 22:23:58.251972 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.251921    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:58.253767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.253734    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.262977 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.262949    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh" pod "9ac46abd-764c-465d-9952-af513f3af401" (UID: "9ac46abd-764c-465d-9952-af513f3af401"). InnerVolumeSpecName "kube-api-access-hdxwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:58.360414 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360357    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxwh\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-kube-api-access-hdxwh\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360452    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360470    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360484    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac46abd-764c-465d-9952-af513f3af401-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360499    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360511    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac46abd-764c-465d-9952-af513f3af401-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360531    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360545    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac46abd-764c-465d-9952-af513f3af401-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.360607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.360556    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac46abd-764c-465d-9952-af513f3af401-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.652008 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.651971    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:58.659180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.659156    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:58.775076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.775040    4536 generic.go:356] "Generic (PLEG): container finished" podID="9ac46abd-764c-465d-9952-af513f3af401" containerID="bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7" exitCode=0
Oct 30 22:23:58.775246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.775102    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"
Oct 30 22:23:58.775246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.775199    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" event={"ID":"9ac46abd-764c-465d-9952-af513f3af401","Type":"ContainerDied","Data":"bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7"}
Oct 30 22:23:58.775359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.775248    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh" event={"ID":"9ac46abd-764c-465d-9952-af513f3af401","Type":"ContainerDied","Data":"6c7565f800cf78ef881d4dc023f9f96ddf2d787100f81eebdca971dfebd831fb"}
Oct 30 22:23:58.775359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.775279    4536 scope.go:122] "RemoveContainer" containerID="bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7"
Oct 30 22:23:58.786042 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.786018    4536 generic.go:356] "Generic (PLEG): container finished" podID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerID="4dc8eb11653e76e7b6ed2f82b565f67b47320498b0c3af05f729466a63242501" exitCode=0
Oct 30 22:23:58.786180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.786146    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" event={"ID":"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7","Type":"ContainerDied","Data":"4dc8eb11653e76e7b6ed2f82b565f67b47320498b0c3af05f729466a63242501"}
Oct 30 22:23:58.786961 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.786943    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:58.788204 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.788184    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:23:58.795197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.795180    4536 scope.go:122] "RemoveContainer" containerID="bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7"
Oct 30 22:23:58.795507 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:23:58.795479    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7\": container with ID starting with bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7 not found: ID does not exist" containerID="bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7"
Oct 30 22:23:58.795633 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.795520    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7"} err="failed to get container status \"bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7\": rpc error: code = NotFound desc = could not find container \"bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7\": container with ID starting with bcb63e2aece36d3631c0c0a236adfe68ab407f2a923d05aae78b4972dff02da7 not found: ID does not exist"
Oct 30 22:23:58.848693 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.848673    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:23:58.971253 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971220    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971282    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971336    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971591 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971485    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971591 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971549    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.971591 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971582    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971619    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971679    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971737    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.971967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971759    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.971967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971833    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.971967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.971874    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs\") pod \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\" (UID: \"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7\") "
Oct 30 22:23:58.972130 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972053    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:58.972337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972191    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:58.972862 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972836    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.972975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972864    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.972975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972879    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.972975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972893    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.972975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.972907    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:58.973970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.973940    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "kube-api-access-t4l8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:58.973970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.973960    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:58.974095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.974062    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:58.974172 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:58.974155    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy" pod "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" (UID: "739cf74d-457c-4bcb-b7ed-64e3ed90aaf7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:59.074502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.074420    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.074502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.074456    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.074502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.074471    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4l8q\" (UniqueName: \"kubernetes.io/projected/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-kube-api-access-t4l8q\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.074502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.074486    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.120971 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.120939    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"]
Oct 30 22:23:59.127062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.127033    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-4bcjh"]
Oct 30 22:23:59.167220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.167186    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ac46abd-764c-465d-9952-af513f3af401" path="/var/lib/kubelet/pods/9ac46abd-764c-465d-9952-af513f3af401/volumes"
Oct 30 22:23:59.167907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.167885    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" path="/var/lib/kubelet/pods/a5c61b4c-c214-4eb0-acbe-6515a319d684/volumes"
Oct 30 22:23:59.795068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.795033    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"
Oct 30 22:23:59.795068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.795052    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv" event={"ID":"739cf74d-457c-4bcb-b7ed-64e3ed90aaf7","Type":"ContainerDied","Data":"685b46ba75265d9774fa0b19db5bedad64cc20c38c74df3ad6cbb678afc74120"}
Oct 30 22:23:59.795337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.795127    4536 scope.go:122] "RemoveContainer" containerID="4dc8eb11653e76e7b6ed2f82b565f67b47320498b0c3af05f729466a63242501"
Oct 30 22:23:59.980715 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:23:59.980682    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"]
Oct 30 22:24:00.031097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:00.031067    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-gdnzv"]
Oct 30 22:24:01.291717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.291680    4536 generic.go:356] "Generic (PLEG): container finished" podID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerID="ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa" exitCode=0
Oct 30 22:24:01.310465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.310431    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" path="/var/lib/kubelet/pods/739cf74d-457c-4bcb-b7ed-64e3ed90aaf7/volumes"
Oct 30 22:24:01.311103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.310931    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" event={"ID":"6a8b6174-67fc-4776-bc8f-4a72565638ff","Type":"ContainerDied","Data":"ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"}
Oct 30 22:24:01.312297 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.312272    4536 scope.go:122] "RemoveContainer" containerID="ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"
Oct 30 22:24:01.316621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.316582    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:01.316715 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.316661    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:01.316715 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:01.316691    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:02.368431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:02.361276    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" event={"ID":"6a8b6174-67fc-4776-bc8f-4a72565638ff","Type":"ContainerStarted","Data":"04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31"}
Oct 30 22:24:02.390787 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:02.389936    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:24:02.391832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:02.391363    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:02.391832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:02.391454    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:03.390807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:03.390340    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:03.390807 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:03.390422    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:04.395422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:04.393823    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:04.395422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:04.393898    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:05.391420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:05.390785    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:05.391420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:05.390847    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:06.391604 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:06.391562    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:06.392709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:06.392676    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:07.510426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:07.480872    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:07.510426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:07.480962    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:08.397419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:08.397220    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:08.397419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:08.397359    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:09.390678 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:09.390637    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 22:24:09.391257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:09.391220    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 22:24:10.219859 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:10.219826    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:24:10.394338 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:10.394310    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:24:10.722528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:10.722501    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:24:13.178345 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178316    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"]
Oct 30 22:24:13.178923 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178909    4536 cpu_manager.go:414] "containerMap: removing container" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy"
Oct 30 22:24:13.178973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178924    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy"
Oct 30 22:24:13.178973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178951    4536 cpu_manager.go:414] "containerMap: removing container" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy"
Oct 30 22:24:13.178973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178957    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy"
Oct 30 22:24:13.178973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178973    4536 cpu_manager.go:414] "containerMap: removing container" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy"
Oct 30 22:24:13.179095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.178978    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy"
Oct 30 22:24:13.179095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.179077    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5c61b4c-c214-4eb0-acbe-6515a319d684" containerName="istio-proxy"
Oct 30 22:24:13.179095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.179086    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="739cf74d-457c-4bcb-b7ed-64e3ed90aaf7" containerName="istio-proxy"
Oct 30 22:24:13.179095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.179095    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ac46abd-764c-465d-9952-af513f3af401" containerName="istio-proxy"
Oct 30 22:24:13.181685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.181666    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.229546 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.229516    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"]
Oct 30 22:24:13.232523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.232496    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.282641 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.282610    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"]
Oct 30 22:24:13.318660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318629    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.318840 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318741    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.318840 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318782    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.318840 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318801    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.318840 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318824    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318873    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.319074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318929    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.318972    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319058    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq2dk\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319088    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319108    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319139    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319181    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319196    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319211    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxrrz\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319239    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319262    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.319288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.319280    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.345013 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.344973    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"]
Oct 30 22:24:13.421037 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421002    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421201 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421107    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq2dk\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421201 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421141    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421201 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421172    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421216    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421267    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421289    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421311    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxrrz\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421518 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421353    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421518 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421388    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421518 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421474    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421518 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421512    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421614    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421677 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421656    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421713 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421687    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421743 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421724    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421782    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.421900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421885    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.421937 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.421894    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.422275 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.422255    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.422581 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.422559    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.423556 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.423532    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.423956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.423927    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.424138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.424116    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.424472 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.424446    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.424816 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.424790    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.425119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.425099    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.425301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.425116    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.425630 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.425607    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.425884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.425861    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.426014 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.425999    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.427458 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.427440    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.487570 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.487537    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.487717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.487620    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq2dk\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.487717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.487633    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.487815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.487796    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxrrz\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz\") pod \"e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") " pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.494180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.494164    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:13.545891 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.545861    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:13.935128 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.935048    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"]
Oct 30 22:24:13.973969 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:24:13.973941    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef96c63_5981_48c0_9791_1ca4e59e2aa7.slice/crio-0543781d58bda2c9f2d0bd0be9c34fc47cd0687b83acbdf43aaafa71f37270c7 WatchSource:0}: Error finding container 0543781d58bda2c9f2d0bd0be9c34fc47cd0687b83acbdf43aaafa71f37270c7: Status 404 returned error can't find the container with id 0543781d58bda2c9f2d0bd0be9c34fc47cd0687b83acbdf43aaafa71f37270c7
Oct 30 22:24:13.976732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.976698    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:13.976815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.976764    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:13.976815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:13.976807    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:14.016726 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:24:14.016700    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9022cf_7a9f_461a_9b3e_eadfa0719e7e.slice/crio-df73cfebbec829b8de0a7d6cde27d0dbb3f2b88888e2ea87b13637da53d64d54 WatchSource:0}: Error finding container df73cfebbec829b8de0a7d6cde27d0dbb3f2b88888e2ea87b13637da53d64d54: Status 404 returned error can't find the container with id df73cfebbec829b8de0a7d6cde27d0dbb3f2b88888e2ea87b13637da53d64d54
Oct 30 22:24:14.017160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.017133    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"]
Oct 30 22:24:14.019815 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.019778    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:14.019884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.019860    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:14.019948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.019899    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:14.846219 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.846170    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" event={"ID":"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e","Type":"ContainerStarted","Data":"414fdefadb6f8e73d6762aa940ec1a23412149f7242d52a659515c33218ed18b"}
Oct 30 22:24:14.846922 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.846888    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" event={"ID":"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e","Type":"ContainerStarted","Data":"df73cfebbec829b8de0a7d6cde27d0dbb3f2b88888e2ea87b13637da53d64d54"}
Oct 30 22:24:14.851114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.851086    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" event={"ID":"3ef96c63-5981-48c0-9791-1ca4e59e2aa7","Type":"ContainerStarted","Data":"1ad3e430a9921e3b4c8ca782b0dda3b4bb0aa183e8a1a8622e34ff17d36567a4"}
Oct 30 22:24:14.871642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:14.871579    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" event={"ID":"3ef96c63-5981-48c0-9791-1ca4e59e2aa7","Type":"ContainerStarted","Data":"0543781d58bda2c9f2d0bd0be9c34fc47cd0687b83acbdf43aaafa71f37270c7"}
Oct 30 22:24:15.167037 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.166747    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podStartSLOduration=2.166728233 podStartE2EDuration="2.166728233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:15.165786435 +0000 UTC m=+1786.153594698" watchObservedRunningTime="2026-10-30 22:24:15.166728233 +0000 UTC m=+1786.154536498"
Oct 30 22:24:15.169609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.169566    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podStartSLOduration=2.169556983 podStartE2EDuration="2.169556983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:15.040139327 +0000 UTC m=+1786.027947587" watchObservedRunningTime="2026-10-30 22:24:15.169556983 +0000 UTC m=+1786.157365246"
Oct 30 22:24:15.499421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.498747    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:15.506422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.502538    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:15.506422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.502598    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:15.552452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.552219    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:15.552452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.552377    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:15.552452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:15.552443    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:16.509428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:16.507512    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:16.509428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:16.507580    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:16.547924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:16.547866    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:16.548104 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:16.547926    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:17.500900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:17.500855    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:17.501198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:17.501170    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:17.551486 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:17.551442    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:17.552034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:17.551517    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:18.510508 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:18.510474    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:18.510763 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:18.510734    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:18.546951 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:18.546833    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:18.546951 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:18.546891    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:19.679196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:19.675673    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:19.679196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:19.675740    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:19.863456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:19.863419    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:19.863724 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:19.863693    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:20.252427 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.252370    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:20.263860 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.254069    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:20.348753 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.348712    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 30 22:24:20.496210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.496174    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:20.496497 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.496465    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:20.546515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.546425    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:20.567315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.567277    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:20.712849 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:20.712737    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 22:24:21.058441 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:21.055539    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:21.058441 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:24:21.057693    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:24:21.266576 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:21.266535    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:21.497509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:21.497447    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:24:21.497880 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:21.497833    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:24:21.552932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:21.552898    4536 patch_prober.go:29] interesting pod/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:24:21.553213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:21.553166    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:24:22.536388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:22.536349    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:22.585131 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:22.578587    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:22.585509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:22.585484    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:24:22.777451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:22.776911    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:22.777962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:22.777936    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:22.779457 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:22.779439    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:24:23.693543 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:23.693494    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:24:23.694132 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:23.693555    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:24:27.087910 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:27.087872    4536 generic.go:356] "Generic (PLEG): container finished" podID="fe027fb4-c600-4b5d-804e-966790379969" containerID="c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0" exitCode=0
Oct 30 22:24:27.088342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:27.087946    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" event={"ID":"fe027fb4-c600-4b5d-804e-966790379969","Type":"ContainerDied","Data":"c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"}
Oct 30 22:24:27.088458 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:27.088441    4536 scope.go:122] "RemoveContainer" containerID="c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"
Oct 30 22:24:27.090252 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:27.090217    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:27.090362 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:27.090302    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:27.090362 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:27.090335    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:24:28.120640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:28.120604    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" event={"ID":"fe027fb4-c600-4b5d-804e-966790379969","Type":"ContainerStarted","Data":"d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430"}
Oct 30 22:24:28.395476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:28.394871    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:24:28.395476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:28.394990    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:24:28.395476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:28.395031    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:24:29.393600 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:29.393562    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:24:29.397538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:29.397501    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:24:30.393476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:30.387449    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:24:30.393476 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:30.387545    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:24:31.396593 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:31.396557    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:24:31.397088 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:31.396633    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:24:32.396865 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:32.396815    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:24:32.397455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:32.396889    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:24:33.377056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:33.377021    4536 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:24:33.377258 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:33.377096    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:24:34.382033 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:34.382001    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:24:34.382509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:34.382496    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:24:34.383248 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:24:34.383228    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:24:50.806225 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:24:50.806184    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:25:20.807385 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:25:20.807294    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:25:34.756139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.756106    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"]
Oct 30 22:25:34.761882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.761856    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.865932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.865902    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.866176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.866048    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.866176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.866098    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms69f\" (UniqueName: \"kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.866176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.866166    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.866465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.866352    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.866465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.866382    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.911118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.911088    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-18515933\"/\"kube-root-ca.crt\""
Oct 30 22:25:34.943749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.943719    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-18515933\"/\"openshift-service-ca.crt\""
Oct 30 22:25:34.968995 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.968947    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-18515933\"/\"default-dockercfg-8v2hp\""
Oct 30 22:25:34.970552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.970486    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.971252 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.971120    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.971769 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.971732    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms69f\" (UniqueName: \"kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.971998 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.971980    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.975247 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.975226    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.975381 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.975356    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.975524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.975505    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.975613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.975578    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.975936 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.975915    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.977004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.976984    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"]
Oct 30 22:25:34.981271 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.981239    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-18515933\"/\"e2e-lifecycle-model-18515933-kserve-self-signed-certs\""
Oct 30 22:25:34.987679 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.984583    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:34.987679 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:34.985668    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:35.050636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:35.050558    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms69f\" (UniqueName: \"kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f\") pod \"e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") " pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:35.078068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:35.078041    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:35.591628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:35.591602    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"]
Oct 30 22:25:35.592471 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:25:35.592445    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfedefc96_6844_46b7_b33f_b06e465914fb.slice/crio-59b11047b953f2451baa533c54c65d8fc0c3c89b18ac39411d04f530ef66182f WatchSource:0}: Error finding container 59b11047b953f2451baa533c54c65d8fc0c3c89b18ac39411d04f530ef66182f: Status 404 returned error can't find the container with id 59b11047b953f2451baa533c54c65d8fc0c3c89b18ac39411d04f530ef66182f
Oct 30 22:25:35.635846 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:35.635817    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld" event={"ID":"fedefc96-6844-46b7-b33f-b06e465914fb","Type":"ContainerStarted","Data":"59b11047b953f2451baa533c54c65d8fc0c3c89b18ac39411d04f530ef66182f"}
Oct 30 22:25:36.644139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:36.644104    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld" event={"ID":"fedefc96-6844-46b7-b33f-b06e465914fb","Type":"ContainerStarted","Data":"129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d"}
Oct 30 22:25:36.770145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:36.770080    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld" podStartSLOduration=2.770059652 podStartE2EDuration="2.770059652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:36.767777399 +0000 UTC m=+1867.755585663" watchObservedRunningTime="2026-10-30 22:25:36.770059652 +0000 UTC m=+1867.757867919"
Oct 30 22:25:37.079141 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:37.079103    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:45.078858 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:45.078825    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:45.177301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:45.177274    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:45.717186 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:25:45.717157    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:25:50.903435 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:25:50.901070    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:26:04.872932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:04.872899    4536 generic.go:356] "Generic (PLEG): container finished" podID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerID="26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818" exitCode=0
Oct 30 22:26:04.873448 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:04.872975    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" event={"ID":"faef18ca-2ab9-4f4d-9611-7e7b49314f18","Type":"ContainerDied","Data":"26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"}
Oct 30 22:26:04.873531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:04.873505    4536 scope.go:122] "RemoveContainer" containerID="26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"
Oct 30 22:26:04.875261 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:04.875231    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:26:04.875332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:04.875300    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:26:04.875369 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:04.875335    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:26:05.881968 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:05.881935    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" event={"ID":"faef18ca-2ab9-4f4d-9611-7e7b49314f18","Type":"ContainerStarted","Data":"4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0"}
Oct 30 22:26:06.651634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:06.651601    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:26:06.651817 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:06.651774    4536 patch_prober.go:29] interesting pod/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:26:06.651869 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:06.651822    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:26:07.652062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:07.652027    4536 patch_prober.go:29] interesting pod/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:26:07.652480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:07.652083    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:26:08.652653 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:08.652616    4536 patch_prober.go:29] interesting pod/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:26:08.653041 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:08.652677    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:26:09.652479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:09.652444    4536 patch_prober.go:29] interesting pod/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:26:09.652660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:09.652515    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:26:10.651687 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:10.651652    4536 patch_prober.go:29] interesting pod/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:26:10.652102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:10.651707    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:26:11.652377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:11.652331    4536 patch_prober.go:29] interesting pod/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:26:11.652876 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:11.652418    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:26:12.657046 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:12.657016    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:26:12.658024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:12.657494    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:26:12.658359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:12.658340    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:26:16.804315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.804263    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"]
Oct 30 22:26:16.830939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.830914    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:16.878805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.876149    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"]
Oct 30 22:26:16.912668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.912640    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a42692-serving-cert\""
Oct 30 22:26:16.936707 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.936665    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:16.936901 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.936725    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpgpf\" (UniqueName: \"kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:16.937088 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.937063    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:16.937163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:16.937126    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.038895 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.038865    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.039097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.038916    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.039097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.038966    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.039097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.038999    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpgpf\" (UniqueName: \"kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.039097 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:26:17.039037    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-serving-cert: secret "maas-api-e2e-api-a-a42692-serving-cert" not found
Oct 30 22:26:17.039318 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:26:17.039114    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls podName:54b62d3f-56ba-4357-8776-e8cafb8e3a3f nodeName:}" failed. No retries permitted until 2026-10-30 22:26:17.539094311 +0000 UTC m=+1908.526902558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls") pod "maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" (UID: "54b62d3f-56ba-4357-8776-e8cafb8e3a3f") : secret "maas-api-e2e-api-a-a42692-serving-cert" not found
Oct 30 22:26:17.039698 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.039675    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.042104 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.042085    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.130075 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.129990    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpgpf\" (UniqueName: \"kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.555336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.555300    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.557912 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.557891    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\") pod \"maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:17.780391 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:17.780357    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:18.165446 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:18.165414    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"]
Oct 30 22:26:18.166033 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:26:18.166006    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b62d3f_56ba_4357_8776_e8cafb8e3a3f.slice/crio-6bd341b90b6651f368ddfa34c4be4edbaf6731632581d2dc91624bccad116db5 WatchSource:0}: Error finding container 6bd341b90b6651f368ddfa34c4be4edbaf6731632581d2dc91624bccad116db5: Status 404 returned error can't find the container with id 6bd341b90b6651f368ddfa34c4be4edbaf6731632581d2dc91624bccad116db5
Oct 30 22:26:18.167612 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:18.167592    4536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:26:18.998820 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:18.998778    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" event={"ID":"54b62d3f-56ba-4357-8776-e8cafb8e3a3f","Type":"ContainerStarted","Data":"6bd341b90b6651f368ddfa34c4be4edbaf6731632581d2dc91624bccad116db5"}
Oct 30 22:26:20.007553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:20.007516    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" event={"ID":"54b62d3f-56ba-4357-8776-e8cafb8e3a3f","Type":"ContainerStarted","Data":"f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba"}
Oct 30 22:26:20.008089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:20.007656    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:20.139583 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:20.139517    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" podStartSLOduration=3.364423345 podStartE2EDuration="4.139495115s" totalImagesPullingTime="775.07177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:26:20.128772537 +0000 UTC m=+1911.116580801" watchObservedRunningTime="2026-10-30 22:26:20.139495115 +0000 UTC m=+1911.127303380"
Oct 30 22:26:20.807192 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:26:20.807160    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:26:24.171932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:24.171900    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"]
Oct 30 22:26:24.172429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:24.172087    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" containerID="cri-o://56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe" gracePeriod=30
Oct 30 22:26:24.513519 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:24.513487    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"]
Oct 30 22:26:24.513723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:24.513704    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy" containerID="cri-o://b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4" gracePeriod=30
Oct 30 22:26:26.021345 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:26.021317    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:26:27.467283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:27.467235    4536 patch_prober.go:29] interesting pod/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:26:27.467718 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:27.467296    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:26:29.475215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.475188    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:26:29.583530 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583495    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.583729 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583606    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.583729 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583642    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.583853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583741    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.583853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583775    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.583853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583811    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.584021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583880    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.584021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.583938    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.584021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.584011    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket\") pod \"13b86d04-856a-484a-8cb4-f9092faf55d7\" (UID: \"13b86d04-856a-484a-8cb4-f9092faf55d7\") "
Oct 30 22:26:29.584165 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.584137    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.584336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.584315    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.584596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.584577    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.584684 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.584618    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.584751 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.584706    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:29.585118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.585027    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.585118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.585054    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.585118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.585068    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.585118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.585082    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b86d04-856a-484a-8cb4-f9092faf55d7-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.585118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.585096    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.587012 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.586987    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:29.587190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.587174    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.587462 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.587445    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:29.587638 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.587610    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86" pod "13b86d04-856a-484a-8cb4-f9092faf55d7" (UID: "13b86d04-856a-484a-8cb4-f9092faf55d7"). InnerVolumeSpecName "kube-api-access-wsz86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:29.687655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.687622    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.687655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.687650    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.687655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.687660    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b86d04-856a-484a-8cb4-f9092faf55d7-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.687963 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.687669    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsz86\" (UniqueName: \"kubernetes.io/projected/13b86d04-856a-484a-8cb4-f9092faf55d7-kube-api-access-wsz86\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.832006 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.831977    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:26:29.890340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890309    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890454    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890512    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890580    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890707 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890659    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890764 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890717    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890871 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890853    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890945 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890871    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.890945 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890915    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.890945 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890894    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.891107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890956    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token\") pod \"06e049f4-7335-4ce8-8fab-e8dfee377277\" (UID: \"06e049f4-7335-4ce8-8fab-e8dfee377277\") "
Oct 30 22:26:29.891107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.890941    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.891107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891014    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.891267 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891205    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:29.891611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891585    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.891738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891613    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.891738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891629    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.891738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891642    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e049f4-7335-4ce8-8fab-e8dfee377277-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.891738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.891656    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.892956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.892935    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:29.893053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.892995    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "kube-api-access-b46s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:29.893195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.893181    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:29.893265 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.893250    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token" pod "06e049f4-7335-4ce8-8fab-e8dfee377277" (UID: "06e049f4-7335-4ce8-8fab-e8dfee377277"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:29.992903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.992803    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.992903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.992841    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.992903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.992857    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e049f4-7335-4ce8-8fab-e8dfee377277-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:29.992903 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:29.992872    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b46s8\" (UniqueName: \"kubernetes.io/projected/06e049f4-7335-4ce8-8fab-e8dfee377277-kube-api-access-b46s8\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:30.085552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.085516    4536 generic.go:356] "Generic (PLEG): container finished" podID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerID="56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe" exitCode=0
Oct 30 22:26:30.085712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.085567    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"
Oct 30 22:26:30.085712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.085594    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" event={"ID":"13b86d04-856a-484a-8cb4-f9092faf55d7","Type":"ContainerDied","Data":"56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe"}
Oct 30 22:26:30.085712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.085649    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw" event={"ID":"13b86d04-856a-484a-8cb4-f9092faf55d7","Type":"ContainerDied","Data":"5b3e007e825d65b5b199453409936b36e8167ea87bf600bb346fddf995b22c54"}
Oct 30 22:26:30.085712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.085671    4536 scope.go:122] "RemoveContainer" containerID="56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe"
Oct 30 22:26:30.087327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.087257    4536 generic.go:356] "Generic (PLEG): container finished" podID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerID="b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4" exitCode=0
Oct 30 22:26:30.087327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.087316    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"
Oct 30 22:26:30.087613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.087337    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" event={"ID":"06e049f4-7335-4ce8-8fab-e8dfee377277","Type":"ContainerDied","Data":"b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4"}
Oct 30 22:26:30.087613 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.087388    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7" event={"ID":"06e049f4-7335-4ce8-8fab-e8dfee377277","Type":"ContainerDied","Data":"1eef3191c77a8c0320a5f8cd83f3b1117ae405da16832b883f562b4db753c889"}
Oct 30 22:26:30.098203 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.098120    4536 scope.go:122] "RemoveContainer" containerID="56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe"
Oct 30 22:26:30.098548 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:26:30.098518    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe\": container with ID starting with 56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe not found: ID does not exist" containerID="56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe"
Oct 30 22:26:30.098624 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.098563    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe"} err="failed to get container status \"56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe\": rpc error: code = NotFound desc = could not find container \"56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe\": container with ID starting with 56bccc8a97cebe52b6283c1b017d78272e27348e4813915ec8184734e659effe not found: ID does not exist"
Oct 30 22:26:30.098624 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.098587    4536 scope.go:122] "RemoveContainer" containerID="b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4"
Oct 30 22:26:30.113736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.113710    4536 scope.go:122] "RemoveContainer" containerID="b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4"
Oct 30 22:26:30.114253 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:26:30.114230    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4\": container with ID starting with b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4 not found: ID does not exist" containerID="b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4"
Oct 30 22:26:30.114349 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.114262    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4"} err="failed to get container status \"b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4\": rpc error: code = NotFound desc = could not find container \"b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4\": container with ID starting with b5fed5b07d220c00e673c5b005d4ee6d8609d676b7856d2c931c3f1042dc92e4 not found: ID does not exist"
Oct 30 22:26:30.259139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.259048    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"]
Oct 30 22:26:30.267178 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.267143    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-wwtq7"]
Oct 30 22:26:30.380111 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.380074    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"]
Oct 30 22:26:30.388292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:30.388265    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-ffjmw"]
Oct 30 22:26:31.166246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:31.166217    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" path="/var/lib/kubelet/pods/06e049f4-7335-4ce8-8fab-e8dfee377277/volumes"
Oct 30 22:26:31.166754 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:31.166739    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" path="/var/lib/kubelet/pods/13b86d04-856a-484a-8cb4-f9092faf55d7/volumes"
Oct 30 22:26:50.807250 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:26:50.807161    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:26:54.263960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:54.263926    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"]
Oct 30 22:26:54.264412 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:54.264216    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy" containerID="cri-o://1ad3e430a9921e3b4c8ca782b0dda3b4bb0aa183e8a1a8622e34ff17d36567a4" gracePeriod=30
Oct 30 22:26:54.277027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:54.276999    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"]
Oct 30 22:26:54.277207 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:54.277187    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy" containerID="cri-o://27bcde5ab3623921fc7dce39f009764e5b6eda64e003000a25d62d915cc017c5" gracePeriod=30
Oct 30 22:26:54.301719 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:54.301690    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"]
Oct 30 22:26:54.301899 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:54.301881    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy" containerID="cri-o://414fdefadb6f8e73d6762aa940ec1a23412149f7242d52a659515c33218ed18b" gracePeriod=30
Oct 30 22:26:59.576501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.576417    4536 generic.go:356] "Generic (PLEG): container finished" podID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerID="414fdefadb6f8e73d6762aa940ec1a23412149f7242d52a659515c33218ed18b" exitCode=0
Oct 30 22:26:59.577011 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.576552    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" event={"ID":"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e","Type":"ContainerDied","Data":"414fdefadb6f8e73d6762aa940ec1a23412149f7242d52a659515c33218ed18b"}
Oct 30 22:26:59.578520 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.578489    4536 generic.go:356] "Generic (PLEG): container finished" podID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerID="27bcde5ab3623921fc7dce39f009764e5b6eda64e003000a25d62d915cc017c5" exitCode=0
Oct 30 22:26:59.578643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.578568    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" event={"ID":"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c","Type":"ContainerDied","Data":"27bcde5ab3623921fc7dce39f009764e5b6eda64e003000a25d62d915cc017c5"}
Oct 30 22:26:59.580224 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.580196    4536 generic.go:356] "Generic (PLEG): container finished" podID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerID="1ad3e430a9921e3b4c8ca782b0dda3b4bb0aa183e8a1a8622e34ff17d36567a4" exitCode=0
Oct 30 22:26:59.580337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.580290    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" event={"ID":"3ef96c63-5981-48c0-9791-1ca4e59e2aa7","Type":"ContainerDied","Data":"1ad3e430a9921e3b4c8ca782b0dda3b4bb0aa183e8a1a8622e34ff17d36567a4"}
Oct 30 22:26:59.626882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.626858    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:26:59.649777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.649750    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:26:59.678525 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.678497    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:26:59.733264 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733231    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.733264 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733268    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.733503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733292    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.733503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733336    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.733503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733364    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.733662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733518    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.733662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733531    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.733662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733587    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.733821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733605    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:59.733821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733705    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.733821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733663    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.733964 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733852    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.733964 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733911    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.734073 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.733997    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.734073 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734034    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.734173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734089    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.734173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734122    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.734173 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734163    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.734318 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734247    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.734318 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734311    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket\") pod \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\" (UID: \"3ef96c63-5981-48c0-9791-1ca4e59e2aa7\") "
Oct 30 22:26:59.734436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734372    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.734497 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734468    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq\") pod \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\" (UID: \"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c\") "
Oct 30 22:26:59.735076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734778    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.735076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.734914    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735304    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735370    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735435    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735454    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735469    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735486    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.735663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.735499    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.736236 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736151    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:59.736236 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736176    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.736451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736341    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.736451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736371    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "kube-api-access-nq2dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:59.736451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736422    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.736647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736455    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:59.736647 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.736569    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.737262 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.737234    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "kube-api-access-hcghq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:59.737423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.737371    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.737736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.737714    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo" pod "3ef96c63-5981-48c0-9791-1ca4e59e2aa7" (UID: "3ef96c63-5981-48c0-9791-1ca4e59e2aa7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:59.738179 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.738161    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo" pod "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" (UID: "30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:59.836094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836006    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.836094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836067    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.836306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836119    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.836306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836169    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.836306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836270    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.836547 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836354    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.836547 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836499    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.836668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836571    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.837195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836752    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.837195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836791    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.837195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836841    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data\") pod \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\" (UID: \"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e\") "
Oct 30 22:26:59.837195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.836965    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:59.837617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837255    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.837617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837587    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.837731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837646    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.837731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837668    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.837731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837682    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.837731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837695    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.837731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837709    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.837731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837724    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837739    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837752    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837767    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837782    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcghq\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-kube-api-access-hcghq\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837796    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837809    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837822    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837835    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq2dk\" (UniqueName: \"kubernetes.io/projected/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-kube-api-access-nq2dk\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837848    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.838043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.837861    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef96c63-5981-48c0-9791-1ca4e59e2aa7-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.839246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.839161    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "kube-api-access-wxrrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:59.839362 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.839247    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:59.839924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.839876    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:59.840171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.840106    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo" pod "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" (UID: "9a9022cf-7a9f-461a-9b3e-eadfa0719e7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:59.939028 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.938998    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.939028 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.939025    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.939028 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.939034    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.939254 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.939044    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxrrz\" (UniqueName: \"kubernetes.io/projected/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-kube-api-access-wxrrz\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:59.939254 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:26:59.939055    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:00.588303 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.588221    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"
Oct 30 22:27:00.588805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.588220    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659" event={"ID":"30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c","Type":"ContainerDied","Data":"8edbb2d69f79d88cb3635bd700c67f365991ccd6673baf8c23d59ce5802d11db"}
Oct 30 22:27:00.588805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.588362    4536 scope.go:122] "RemoveContainer" containerID="27bcde5ab3623921fc7dce39f009764e5b6eda64e003000a25d62d915cc017c5"
Oct 30 22:27:00.589789 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.589753    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"
Oct 30 22:27:00.589857 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.589781    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw" event={"ID":"3ef96c63-5981-48c0-9791-1ca4e59e2aa7","Type":"ContainerDied","Data":"0543781d58bda2c9f2d0bd0be9c34fc47cd0687b83acbdf43aaafa71f37270c7"}
Oct 30 22:27:00.591385 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.591360    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2" event={"ID":"9a9022cf-7a9f-461a-9b3e-eadfa0719e7e","Type":"ContainerDied","Data":"df73cfebbec829b8de0a7d6cde27d0dbb3f2b88888e2ea87b13637da53d64d54"}
Oct 30 22:27:00.591529 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.591416    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"
Oct 30 22:27:00.600836 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.600819    4536 scope.go:122] "RemoveContainer" containerID="1ad3e430a9921e3b4c8ca782b0dda3b4bb0aa183e8a1a8622e34ff17d36567a4"
Oct 30 22:27:00.611169 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.611147    4536 scope.go:122] "RemoveContainer" containerID="414fdefadb6f8e73d6762aa940ec1a23412149f7242d52a659515c33218ed18b"
Oct 30 22:27:00.758507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.758469    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"]
Oct 30 22:27:00.808117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.808082    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-sv659"]
Oct 30 22:27:00.873220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.873127    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"]
Oct 30 22:27:00.878697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.878662    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-thwl2"]
Oct 30 22:27:00.993811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:00.993754    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"]
Oct 30 22:27:01.009193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:01.008976    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-367f4491-openshift-default-5f467bc7d9-g4nmw"]
Oct 30 22:27:01.242421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:01.240782    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" path="/var/lib/kubelet/pods/30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c/volumes"
Oct 30 22:27:01.275849 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:01.275811    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" path="/var/lib/kubelet/pods/3ef96c63-5981-48c0-9791-1ca4e59e2aa7/volumes"
Oct 30 22:27:01.277026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:01.276999    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" path="/var/lib/kubelet/pods/9a9022cf-7a9f-461a-9b3e-eadfa0719e7e/volumes"
Oct 30 22:27:08.452233 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:08.452197    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"]
Oct 30 22:27:08.452686 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:08.452418    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" containerID="cri-o://13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f" gracePeriod=30
Oct 30 22:27:12.062420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:12.056462    4536 patch_prober.go:29] interesting pod/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:27:12.062420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:12.056569    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:27:13.721199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.721138    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:27:13.885009 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.884980    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885035    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885106    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885192    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885245    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885286    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885439    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:13.885503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885447    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885535    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885547    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:13.885611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885558    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:13.885758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885611    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs\") pod \"1d307b9c-aae8-46ed-b511-8094d566bd45\" (UID: \"1d307b9c-aae8-46ed-b511-8094d566bd45\") "
Oct 30 22:27:13.885812 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885757    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:13.886029 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.885853    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:13.886210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.886190    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d307b9c-aae8-46ed-b511-8094d566bd45-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.886299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.886210    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.886299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.886219    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.886299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.886227    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.886299 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.886236    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.887620 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.887594    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:27:13.887620 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.887606    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:13.887741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.887714    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "kube-api-access-t4xhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:13.888443 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.888424    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token" pod "1d307b9c-aae8-46ed-b511-8094d566bd45" (UID: "1d307b9c-aae8-46ed-b511-8094d566bd45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:13.988208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.988125    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.988208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.988158    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.988208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.988169    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4xhn\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-kube-api-access-t4xhn\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:13.988208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:13.988179    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d307b9c-aae8-46ed-b511-8094d566bd45-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:14.089307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.089274    4536 generic.go:356] "Generic (PLEG): container finished" podID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerID="13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f" exitCode=0
Oct 30 22:27:14.089500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.089330    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"
Oct 30 22:27:14.089500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.089360    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" event={"ID":"1d307b9c-aae8-46ed-b511-8094d566bd45","Type":"ContainerDied","Data":"13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f"}
Oct 30 22:27:14.089500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.089480    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c" event={"ID":"1d307b9c-aae8-46ed-b511-8094d566bd45","Type":"ContainerDied","Data":"92b4898ae0238229761dde6322a37522eb7d5c382c092cae237668dc042aab01"}
Oct 30 22:27:14.089611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.089509    4536 scope.go:122] "RemoveContainer" containerID="13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f"
Oct 30 22:27:14.100469 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.100452    4536 scope.go:122] "RemoveContainer" containerID="13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f"
Oct 30 22:27:14.100770 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:27:14.100750    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f\": container with ID starting with 13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f not found: ID does not exist" containerID="13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f"
Oct 30 22:27:14.100855 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.100778    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f"} err="failed to get container status \"13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f\": rpc error: code = NotFound desc = could not find container \"13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f\": container with ID starting with 13387f6e1428f69f07d8d1ecb59c705d98fb1d415c3d3b5e602fac78a02d5b3f not found: ID does not exist"
Oct 30 22:27:14.281477 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.275791    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"]
Oct 30 22:27:14.317169 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:14.313453    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-h499c"]
Oct 30 22:27:15.166802 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:15.166773    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" path="/var/lib/kubelet/pods/1d307b9c-aae8-46ed-b511-8094d566bd45/volumes"
Oct 30 22:27:20.806412 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:27:20.806356    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:27:35.804627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:35.804590    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"]
Oct 30 22:27:35.911915 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:35.911883    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"]
Oct 30 22:27:40.805497 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:40.805452    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh" podUID="fc894bbd-6a33-47e0-a429-0e88deb98485" containerName="payload-processing" containerID="cri-o://761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6" gracePeriod=25
Oct 30 22:27:40.912837 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:40.912783    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v" podUID="be96df3d-6aa7-4826-bacb-bf548e0beca8" containerName="payload-pre-processing" containerID="cri-o://4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd" gracePeriod=25
Oct 30 22:27:41.087684 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.087657    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:27:41.191204 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.191177    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:27:41.215320 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.215294    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79\" (UniqueName: \"kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79\") pod \"fc894bbd-6a33-47e0-a429-0e88deb98485\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") "
Oct 30 22:27:41.215500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.215349    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume\") pod \"fc894bbd-6a33-47e0-a429-0e88deb98485\" (UID: \"fc894bbd-6a33-47e0-a429-0e88deb98485\") "
Oct 30 22:27:41.216673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.216641    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume" pod "fc894bbd-6a33-47e0-a429-0e88deb98485" (UID: "fc894bbd-6a33-47e0-a429-0e88deb98485"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:41.217913 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.217883    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79" pod "fc894bbd-6a33-47e0-a429-0e88deb98485" (UID: "fc894bbd-6a33-47e0-a429-0e88deb98485"). InnerVolumeSpecName "kube-api-access-thn79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:41.317163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.317080    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm\" (UniqueName: \"kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm\") pod \"be96df3d-6aa7-4826-bacb-bf548e0beca8\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") "
Oct 30 22:27:41.317310 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.317261    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume\") pod \"be96df3d-6aa7-4826-bacb-bf548e0beca8\" (UID: \"be96df3d-6aa7-4826-bacb-bf548e0beca8\") "
Oct 30 22:27:41.317774 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.317738    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume" pod "be96df3d-6aa7-4826-bacb-bf548e0beca8" (UID: "be96df3d-6aa7-4826-bacb-bf548e0beca8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:41.318064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.318045    4536 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be96df3d-6aa7-4826-bacb-bf548e0beca8-plugins-config-volume\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:41.318115 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.318071    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thn79\" (UniqueName: \"kubernetes.io/projected/fc894bbd-6a33-47e0-a429-0e88deb98485-kube-api-access-thn79\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:41.318115 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.318086    4536 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc894bbd-6a33-47e0-a429-0e88deb98485-plugins-config-volume\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:41.319507 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.319484    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm" pod "be96df3d-6aa7-4826-bacb-bf548e0beca8" (UID: "be96df3d-6aa7-4826-bacb-bf548e0beca8"). InnerVolumeSpecName "kube-api-access-9tkbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:41.419801 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.419764    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tkbm\" (UniqueName: \"kubernetes.io/projected/be96df3d-6aa7-4826-bacb-bf548e0beca8-kube-api-access-9tkbm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:41.567639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.567545    4536 generic.go:356] "Generic (PLEG): container finished" podID="be96df3d-6aa7-4826-bacb-bf548e0beca8" containerID="4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd" exitCode=0
Oct 30 22:27:41.567639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.567623    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"
Oct 30 22:27:41.567863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.567636    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v" event={"ID":"be96df3d-6aa7-4826-bacb-bf548e0beca8","Type":"ContainerDied","Data":"4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd"}
Oct 30 22:27:41.567863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.567721    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v" event={"ID":"be96df3d-6aa7-4826-bacb-bf548e0beca8","Type":"ContainerDied","Data":"41b8749ae29d5998b9e075199955e282e38134a78d520a88ab2adeb689496306"}
Oct 30 22:27:41.567863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.567749    4536 scope.go:122] "RemoveContainer" containerID="4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd"
Oct 30 22:27:41.568970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.568954    4536 generic.go:356] "Generic (PLEG): container finished" podID="fc894bbd-6a33-47e0-a429-0e88deb98485" containerID="761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6" exitCode=0
Oct 30 22:27:41.569084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.569016    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"
Oct 30 22:27:41.569084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.569043    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh" event={"ID":"fc894bbd-6a33-47e0-a429-0e88deb98485","Type":"ContainerDied","Data":"761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6"}
Oct 30 22:27:41.569206 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.569087    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh" event={"ID":"fc894bbd-6a33-47e0-a429-0e88deb98485","Type":"ContainerDied","Data":"69068a2f406bd827649d6537b481ba77fbd0a5a5c0da868db071de4dc2b0068d"}
Oct 30 22:27:41.581732 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.581714    4536 scope.go:122] "RemoveContainer" containerID="4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd"
Oct 30 22:27:41.582083 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:27:41.582057    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd\": container with ID starting with 4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd not found: ID does not exist" containerID="4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd"
Oct 30 22:27:41.582228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.582093    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd"} err="failed to get container status \"4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd\": rpc error: code = NotFound desc = could not find container \"4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd\": container with ID starting with 4784ba9478638d6748eb2217b1adc0b94f313e194d4dacc1e7c542aa0f8f4cdd not found: ID does not exist"
Oct 30 22:27:41.582228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.582115    4536 scope.go:122] "RemoveContainer" containerID="761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6"
Oct 30 22:27:41.593176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.593160    4536 scope.go:122] "RemoveContainer" containerID="761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6"
Oct 30 22:27:41.593465 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:27:41.593446    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6\": container with ID starting with 761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6 not found: ID does not exist" containerID="761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6"
Oct 30 22:27:41.593521 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.593474    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6"} err="failed to get container status \"761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6\": rpc error: code = NotFound desc = could not find container \"761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6\": container with ID starting with 761b2b0aff9e63a6e31fcb766dcc3423278aa455b0ed667c447962e1181f2da6 not found: ID does not exist"
Oct 30 22:27:41.735770 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.735740    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"]
Oct 30 22:27:41.785457 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.785423    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2dd57a-6dcb8f5b79-mjp7v"]
Oct 30 22:27:41.855738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.855703    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"]
Oct 30 22:27:41.858840 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:41.858814    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2dd57a-f6d9bbdf9-4r7hh"]
Oct 30 22:27:43.166415 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:43.166374    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be96df3d-6aa7-4826-bacb-bf548e0beca8" path="/var/lib/kubelet/pods/be96df3d-6aa7-4826-bacb-bf548e0beca8/volumes"
Oct 30 22:27:43.166827 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:27:43.166812    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc894bbd-6a33-47e0-a429-0e88deb98485" path="/var/lib/kubelet/pods/fc894bbd-6a33-47e0-a429-0e88deb98485/volumes"
Oct 30 22:27:50.806564 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:27:50.806522    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:28:12.262475 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.262378    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7438-36f5d1"]
Oct 30 22:28:12.262975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.262961    4536 cpu_manager.go:414] "containerMap: removing container" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy"
Oct 30 22:28:12.263023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.262976    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy"
Oct 30 22:28:12.263023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.262991    4536 cpu_manager.go:414] "containerMap: removing container" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy"
Oct 30 22:28:12.263023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.262996    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy"
Oct 30 22:28:12.263023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263011    4536 cpu_manager.go:414] "containerMap: removing container" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy"
Oct 30 22:28:12.263023 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263016    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263025    4536 cpu_manager.go:414] "containerMap: removing container" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263029    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263042    4536 cpu_manager.go:414] "containerMap: removing container" podUID="be96df3d-6aa7-4826-bacb-bf548e0beca8" containerName="payload-pre-processing"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263047    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be96df3d-6aa7-4826-bacb-bf548e0beca8" containerName="payload-pre-processing"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263060    4536 cpu_manager.go:414] "containerMap: removing container" podUID="fc894bbd-6a33-47e0-a429-0e88deb98485" containerName="payload-processing"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263064    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc894bbd-6a33-47e0-a429-0e88deb98485" containerName="payload-processing"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263079    4536 cpu_manager.go:414] "containerMap: removing container" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263083    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263101    4536 cpu_manager.go:414] "containerMap: removing container" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263106    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy"
Oct 30 22:28:12.263197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263198    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="be96df3d-6aa7-4826-bacb-bf548e0beca8" containerName="payload-pre-processing"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263207    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc894bbd-6a33-47e0-a429-0e88deb98485" containerName="payload-processing"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263216    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d307b9c-aae8-46ed-b511-8094d566bd45" containerName="istio-proxy"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263224    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="13b86d04-856a-484a-8cb4-f9092faf55d7" containerName="istio-proxy"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263231    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="30cbd0cb-0e32-41e1-ba9d-013b5f3c2c3c" containerName="istio-proxy"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263238    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a9022cf-7a9f-461a-9b3e-eadfa0719e7e" containerName="istio-proxy"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263245    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ef96c63-5981-48c0-9791-1ca4e59e2aa7" containerName="istio-proxy"
Oct 30 22:28:12.263555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.263250    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="06e049f4-7335-4ce8-8fab-e8dfee377277" containerName="istio-proxy"
Oct 30 22:28:12.266773 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.266759    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:12.303167 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.303128    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7mm5\" (UniqueName: \"kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5\") pod \"test-curl-7438-36f5d1\" (UID: \"35c34740-4bf6-4854-8489-35f702805f5a\") " pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:12.322273 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.322244    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7438-36f5d1"]
Oct 30 22:28:12.367941 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.367913    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tmcpb\""
Oct 30 22:28:12.404676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.404644    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7mm5\" (UniqueName: \"kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5\") pod \"test-curl-7438-36f5d1\" (UID: \"35c34740-4bf6-4854-8489-35f702805f5a\") " pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:12.467172 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.467136    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7mm5\" (UniqueName: \"kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5\") pod \"test-curl-7438-36f5d1\" (UID: \"35c34740-4bf6-4854-8489-35f702805f5a\") " pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:12.579629 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.579560    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:12.960793 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:12.960763    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7438-36f5d1"]
Oct 30 22:28:12.961818 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:28:12.961793    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35c34740_4bf6_4854_8489_35f702805f5a.slice/crio-d8145d03d0c4c4c958caee2580135f916d7b6abeb9cdd6262e88dcd9ac6630b1 WatchSource:0}: Error finding container d8145d03d0c4c4c958caee2580135f916d7b6abeb9cdd6262e88dcd9ac6630b1: Status 404 returned error can't find the container with id d8145d03d0c4c4c958caee2580135f916d7b6abeb9cdd6262e88dcd9ac6630b1
Oct 30 22:28:13.784308 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:13.784269    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7438-36f5d1" event={"ID":"35c34740-4bf6-4854-8489-35f702805f5a","Type":"ContainerStarted","Data":"6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec"}
Oct 30 22:28:13.784308 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:13.784317    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7438-36f5d1" event={"ID":"35c34740-4bf6-4854-8489-35f702805f5a","Type":"ContainerStarted","Data":"d8145d03d0c4c4c958caee2580135f916d7b6abeb9cdd6262e88dcd9ac6630b1"}
Oct 30 22:28:13.900179 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:13.900122    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7438-36f5d1" podStartSLOduration=1.900107504 podStartE2EDuration="1.900107504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:13.898735664 +0000 UTC m=+2024.886543929" watchObservedRunningTime="2026-10-30 22:28:13.900107504 +0000 UTC m=+2024.887915766"
Oct 30 22:28:15.450141 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:15.450101    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7438-36f5d1"]
Oct 30 22:28:15.456699 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:15.456667    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7438-36f5d1"]
Oct 30 22:28:15.811480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:15.809497    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7438-36f5d1" podUID="35c34740-4bf6-4854-8489-35f702805f5a" containerName="test-curl-7438-36f5d1" containerID="cri-o://6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec" gracePeriod=2
Oct 30 22:28:15.860825 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:15.860776    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7438-36f5d1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="35c34740-4bf6-4854-8489-35f702805f5a" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:18.083899 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.083840    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:18.207090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.207056    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5\" (UniqueName: \"kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5\") pod \"35c34740-4bf6-4854-8489-35f702805f5a\" (UID: \"35c34740-4bf6-4854-8489-35f702805f5a\") "
Oct 30 22:28:18.209428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.209371    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5" pod "35c34740-4bf6-4854-8489-35f702805f5a" (UID: "35c34740-4bf6-4854-8489-35f702805f5a"). InnerVolumeSpecName "kube-api-access-s7mm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:18.309226 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.309195    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7mm5\" (UniqueName: \"kubernetes.io/projected/35c34740-4bf6-4854-8489-35f702805f5a-kube-api-access-s7mm5\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:18.831923 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.831886    4536 generic.go:356] "Generic (PLEG): container finished" podID="35c34740-4bf6-4854-8489-35f702805f5a" containerID="6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec" exitCode=137
Oct 30 22:28:18.832098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.831934    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:18.832098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.832020    4536 scope.go:122] "RemoveContainer" containerID="6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec"
Oct 30 22:28:18.842156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.842134    4536 scope.go:122] "RemoveContainer" containerID="6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec"
Oct 30 22:28:18.842539 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:18.842511    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec\": container with ID starting with 6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec not found: ID does not exist" containerID="6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec"
Oct 30 22:28:18.842643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:18.842550    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec"} err="failed to get container status \"6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec\": rpc error: code = NotFound desc = could not find container \"6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec\": container with ID starting with 6b5509692c9eb2b4ca1d1e22f7b7812e8630d27dfcde146e99e7481a72b40cec not found: ID does not exist"
Oct 30 22:28:18.894714 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:18.894672    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7438-36f5d1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="35c34740-4bf6-4854-8489-35f702805f5a" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:19.166309 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:19.166236    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35c34740-4bf6-4854-8489-35f702805f5a" path="/var/lib/kubelet/pods/35c34740-4bf6-4854-8489-35f702805f5a/volumes"
Oct 30 22:28:19.213079 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:19.213046    4536 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7438-36f5d1\" is forbidden: User \"system:node:ip-10-0-138-166.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-166.us-west-1.compute.internal' and this object" podUID="35c34740-4bf6-4854-8489-35f702805f5a" pod="opendatahub/test-curl-7438-36f5d1"
Oct 30 22:28:20.806788 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:20.806754    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:28:24.739300 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:24.739270    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"]
Oct 30 22:28:24.739833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:24.739555    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" containerID="cri-o://0a99d6abc8e07e4a49f187fcb905d9e2a8f4ec91279ab8237942a64aae7d5090" gracePeriod=30
Oct 30 22:28:25.798130 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:25.798096    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"]
Oct 30 22:28:25.798640 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:25.798350    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" containerID="cri-o://31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d" gracePeriod=30
Oct 30 22:28:26.066312 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:26.066232    4536 patch_prober.go:29] interesting pod/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:28:26.066312 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:26.066296    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:28:28.509924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:28.509894    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:28:28.510305 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:28.509945    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:28:29.904114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:29.904075    4536 generic.go:356] "Generic (PLEG): container finished" podID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerID="0a99d6abc8e07e4a49f187fcb905d9e2a8f4ec91279ab8237942a64aae7d5090" exitCode=0
Oct 30 22:28:29.904517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:29.904149    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" event={"ID":"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341","Type":"ContainerDied","Data":"0a99d6abc8e07e4a49f187fcb905d9e2a8f4ec91279ab8237942a64aae7d5090"}
Oct 30 22:28:30.025208 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.025149    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:28:30.105493 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105462    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.105694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105569    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.105694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105608    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.105694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105663    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.105866 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105748    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.105866 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105789    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.105866 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105828    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.106001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105911    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:30.106001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105935    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:30.106001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.105982    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.106145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106080    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo\") pod \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\" (UID: \"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341\") "
Oct 30 22:28:30.106196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106166    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:30.106196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106170    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:30.106314 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106285    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:30.106949 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106921    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.107078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106954    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.107078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106970    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.107078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106985    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.107078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.106998    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.108391 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.108367    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:28:30.108506 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.108381    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:30.108832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.108732    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "kube-api-access-dhgfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:30.108832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.108808    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token" pod "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" (UID: "fa8a888c-12d6-4b7a-8a8b-c63cb09b2341"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:30.207109 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.207078    4536 scope.go:122] "RemoveContainer" containerID="0a99d6abc8e07e4a49f187fcb905d9e2a8f4ec91279ab8237942a64aae7d5090"
Oct 30 22:28:30.208439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.208361    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.208439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.208382    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.208439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.208412    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhgfr\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-kube-api-access-dhgfr\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.208439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.208427    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:30.910048 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.910008    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl" event={"ID":"fa8a888c-12d6-4b7a-8a8b-c63cb09b2341","Type":"ContainerDied","Data":"ff948e323a27eed9e61e4c91d58419d1ac825d2429bf9c7481940f75f6fc6221"}
Oct 30 22:28:30.910489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:30.910067    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"
Oct 30 22:28:31.060909 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.060846    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:28:31.127025 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.126992    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127045    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127093    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127179    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127333 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127303    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127348    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127479 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127423    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127475    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127513    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs\") pod \"7d209b30-5473-4ee4-aa77-395fcaf10225\" (UID: \"7d209b30-5473-4ee4-aa77-395fcaf10225\") "
Oct 30 22:28:31.127629 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127526    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:31.127844 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127780    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:31.127844 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127723    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:31.128194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.127975    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:31.128194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.128100    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:31.128194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.128166    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.128194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.128187    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d209b30-5473-4ee4-aa77-395fcaf10225-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.128436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.128201    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.128436 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.128209    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.129617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.129596    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:31.129709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.129684    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:28:31.129946 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.129925    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:31.129946 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.129940    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct" pod "7d209b30-5473-4ee4-aa77-395fcaf10225" (UID: "7d209b30-5473-4ee4-aa77-395fcaf10225"). InnerVolumeSpecName "kube-api-access-mkxct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:31.230337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.230306    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.230337 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.230337    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.230621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.230350    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.230621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.230358    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkxct\" (UniqueName: \"kubernetes.io/projected/7d209b30-5473-4ee4-aa77-395fcaf10225-kube-api-access-mkxct\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.230621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.230367    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d209b30-5473-4ee4-aa77-395fcaf10225-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:31.319892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.319811    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"]
Oct 30 22:28:31.325439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.325412    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-tfjtl"]
Oct 30 22:28:31.917620 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.917581    4536 generic.go:356] "Generic (PLEG): container finished" podID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerID="31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d" exitCode=0
Oct 30 22:28:31.918027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.917633    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"
Oct 30 22:28:31.918027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.917660    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" event={"ID":"7d209b30-5473-4ee4-aa77-395fcaf10225","Type":"ContainerDied","Data":"31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d"}
Oct 30 22:28:31.918027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.917719    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw" event={"ID":"7d209b30-5473-4ee4-aa77-395fcaf10225","Type":"ContainerDied","Data":"9f68f45c4ce9f5be8761ee116020de37e48efc41ab9cf7ced5066f440acf1b4f"}
Oct 30 22:28:31.918027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.917741    4536 scope.go:122] "RemoveContainer" containerID="31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d"
Oct 30 22:28:31.927274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.927256    4536 scope.go:122] "RemoveContainer" containerID="31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d"
Oct 30 22:28:31.927598 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:31.927578    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d\": container with ID starting with 31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d not found: ID does not exist" containerID="31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d"
Oct 30 22:28:31.927684 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:31.927607    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d"} err="failed to get container status \"31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d\": rpc error: code = NotFound desc = could not find container \"31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d\": container with ID starting with 31ef766317d9e61e526f36fa76e0746548402edb8b6c1165f1a6e6ffeafe1b2d not found: ID does not exist"
Oct 30 22:28:32.086639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:32.086605    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"]
Oct 30 22:28:32.136603 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:32.136574    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-dj2sw"]
Oct 30 22:28:33.165770 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.165737    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" path="/var/lib/kubelet/pods/7d209b30-5473-4ee4-aa77-395fcaf10225/volumes"
Oct 30 22:28:33.166210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.166196    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" path="/var/lib/kubelet/pods/fa8a888c-12d6-4b7a-8a8b-c63cb09b2341/volumes"
Oct 30 22:28:33.782045 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782016    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"]
Oct 30 22:28:33.782602 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782588    4536 cpu_manager.go:414] "containerMap: removing container" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy"
Oct 30 22:28:33.782683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782603    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy"
Oct 30 22:28:33.782683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782640    4536 cpu_manager.go:414] "containerMap: removing container" podUID="35c34740-4bf6-4854-8489-35f702805f5a" containerName="test-curl-7438-36f5d1"
Oct 30 22:28:33.782683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782645    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35c34740-4bf6-4854-8489-35f702805f5a" containerName="test-curl-7438-36f5d1"
Oct 30 22:28:33.782683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782657    4536 cpu_manager.go:414] "containerMap: removing container" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy"
Oct 30 22:28:33.782683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782662    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy"
Oct 30 22:28:33.782837 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782747    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d209b30-5473-4ee4-aa77-395fcaf10225" containerName="istio-proxy"
Oct 30 22:28:33.782837 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782757    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa8a888c-12d6-4b7a-8a8b-c63cb09b2341" containerName="istio-proxy"
Oct 30 22:28:33.782837 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.782769    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="35c34740-4bf6-4854-8489-35f702805f5a" containerName="test-curl-7438-36f5d1"
Oct 30 22:28:33.787758 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.787742    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.863887 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.863845    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98v26\" (UniqueName: \"kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.864094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.863917    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.864094 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.864060    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.864228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.864121    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.891349 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.891321    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-56c628-serving-cert\""
Oct 30 22:28:33.894557 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.894529    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"]
Oct 30 22:28:33.965005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.964969    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.965189 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.965045    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.965189 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.965095    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.965314 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:33.965231    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-serving-cert: secret "maas-api-e2e-api-a-56c628-serving-cert" not found
Oct 30 22:28:33.965314 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:33.965300    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls podName:70861c3f-0093-47d0-8f26-3f7967f40eb5 nodeName:}" failed. No retries permitted until 2026-10-30 22:28:34.465278589 +0000 UTC m=+2045.453086847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls") pod "maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" (UID: "70861c3f-0093-47d0-8f26-3f7967f40eb5") : secret "maas-api-e2e-api-a-56c628-serving-cert" not found
Oct 30 22:28:33.965466 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.965446    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98v26\" (UniqueName: \"kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.965655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.965633    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:33.967778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:33.967756    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:34.062475 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:34.059118    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98v26\" (UniqueName: \"kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:34.499748 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:34.499717    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:34.502293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:34.502273    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\") pod \"maas-api-e2e-api-a-56c628-5b779b7497-9hdkc\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:34.701969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:34.701938    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:35.086628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:35.086597    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"]
Oct 30 22:28:35.087714 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:28:35.087689    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70861c3f_0093_47d0_8f26_3f7967f40eb5.slice/crio-4782829ca91ac704d2a5e965b89fbf5b946b018d342f2c21539c6d1051d23484 WatchSource:0}: Error finding container 4782829ca91ac704d2a5e965b89fbf5b946b018d342f2c21539c6d1051d23484: Status 404 returned error can't find the container with id 4782829ca91ac704d2a5e965b89fbf5b946b018d342f2c21539c6d1051d23484
Oct 30 22:28:35.951422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:35.951316    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" event={"ID":"70861c3f-0093-47d0-8f26-3f7967f40eb5","Type":"ContainerStarted","Data":"4782829ca91ac704d2a5e965b89fbf5b946b018d342f2c21539c6d1051d23484"}
Oct 30 22:28:36.997890 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:36.997851    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" event={"ID":"70861c3f-0093-47d0-8f26-3f7967f40eb5","Type":"ContainerStarted","Data":"974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40"}
Oct 30 22:28:36.999489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:36.999467    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:37.118683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:37.118630    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" podStartSLOduration=3.30823814 podStartE2EDuration="4.118611667s" totalImagesPullingTime="810.373527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:37.117509665 +0000 UTC m=+2048.105317940" watchObservedRunningTime="2026-10-30 22:28:37.118611667 +0000 UTC m=+2048.106419932"
Oct 30 22:28:40.144975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.144943    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"]
Oct 30 22:28:40.149567 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.149546    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.203159 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.203127    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6d4d3c-serving-cert\""
Oct 30 22:28:40.237768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.237727    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbq5p\" (UniqueName: \"kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.237933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.237807    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.237933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.237859    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.238047 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.238011    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.256224 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.256195    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"]
Oct 30 22:28:40.339055 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.339018    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbq5p\" (UniqueName: \"kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.339219 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.339110    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.339219 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.339138    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.339335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.339288    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.339388 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:40.339349    4536 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-serving-cert: secret "maas-api-e2e-api-a-6d4d3c-serving-cert" not found
Oct 30 22:28:40.339450 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:40.339444    4536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls podName:6fed049f-6d70-47db-aeec-2bcbf2271ce9 nodeName:}" failed. No retries permitted until 2026-10-30 22:28:40.839427169 +0000 UTC m=+2051.827235427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls") pod "maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" (UID: "6fed049f-6d70-47db-aeec-2bcbf2271ce9") : secret "maas-api-e2e-api-a-6d4d3c-serving-cert" not found
Oct 30 22:28:40.339985 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.339906    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.341722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.341703    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.462426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.462174    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbq5p\" (UniqueName: \"kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.856387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.856302    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:40.858863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:40.858840    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\") pod \"maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:41.067360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:41.067325    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:41.449930 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:41.449901    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"]
Oct 30 22:28:41.451031 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:28:41.451005    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fed049f_6d70_47db_aeec_2bcbf2271ce9.slice/crio-f0a56edac60a10b39196066d7899a13947c39bf748ac06d074f4fd29a3fb1fdc WatchSource:0}: Error finding container f0a56edac60a10b39196066d7899a13947c39bf748ac06d074f4fd29a3fb1fdc: Status 404 returned error can't find the container with id f0a56edac60a10b39196066d7899a13947c39bf748ac06d074f4fd29a3fb1fdc
Oct 30 22:28:42.049832 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:42.049796    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" event={"ID":"6fed049f-6d70-47db-aeec-2bcbf2271ce9","Type":"ContainerStarted","Data":"f0a56edac60a10b39196066d7899a13947c39bf748ac06d074f4fd29a3fb1fdc"}
Oct 30 22:28:43.057879 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:43.057839    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" event={"ID":"6fed049f-6d70-47db-aeec-2bcbf2271ce9","Type":"ContainerStarted","Data":"86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d"}
Oct 30 22:28:43.058502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:43.057904    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:43.180475 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:43.180419    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" podStartSLOduration=2.381425413 podStartE2EDuration="3.180404208s" totalImagesPullingTime="798.978795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:43.1770143 +0000 UTC m=+2054.164822593" watchObservedRunningTime="2026-10-30 22:28:43.180404208 +0000 UTC m=+2054.168212462"
Oct 30 22:28:44.035451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:44.035423    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:28:49.069592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:49.069560    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:28:50.446913 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:50.446874    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"]
Oct 30 22:28:50.447345 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:50.447143    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy" containerID="cri-o://d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430" gracePeriod=30
Oct 30 22:28:50.807144 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:50.807055    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:28:55.723166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.723091    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:28:55.830424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830363    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830487    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830529    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830572    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830603    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830824 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830683    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830824 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830710    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830819    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:55.830934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830825    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:55.830934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830861    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.830934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.830905    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket\") pod \"fe027fb4-c600-4b5d-804e-966790379969\" (UID: \"fe027fb4-c600-4b5d-804e-966790379969\") "
Oct 30 22:28:55.831205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831091    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:55.831205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831167    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:55.831376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831355    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:55.831632 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831539    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.831632 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831558    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.831632 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831566    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.831632 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831576    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe027fb4-c600-4b5d-804e-966790379969-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.831632 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.831584    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.833274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.833240    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:28:55.833274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.833250    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:55.833497 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.833479    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:28:55.833556 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.833516    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22" pod "fe027fb4-c600-4b5d-804e-966790379969" (UID: "fe027fb4-c600-4b5d-804e-966790379969"). InnerVolumeSpecName "kube-api-access-z5p22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:55.946484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.941465    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe027fb4-c600-4b5d-804e-966790379969-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.946484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.941503    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5p22\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-kube-api-access-z5p22\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.946484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.941520    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe027fb4-c600-4b5d-804e-966790379969-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:55.946484 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:55.941533    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe027fb4-c600-4b5d-804e-966790379969-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:56.142387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.142303    4536 generic.go:356] "Generic (PLEG): container finished" podID="fe027fb4-c600-4b5d-804e-966790379969" containerID="d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430" exitCode=0
Oct 30 22:28:56.142387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.142370    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"
Oct 30 22:28:56.142387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.142383    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" event={"ID":"fe027fb4-c600-4b5d-804e-966790379969","Type":"ContainerDied","Data":"d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430"}
Oct 30 22:28:56.142650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.142499    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk" event={"ID":"fe027fb4-c600-4b5d-804e-966790379969","Type":"ContainerDied","Data":"0f1d2a5a0267267f538fe8f8052663d5b02a43381ea800825ce5c4440df9620a"}
Oct 30 22:28:56.142650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.142522    4536 scope.go:122] "RemoveContainer" containerID="d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430"
Oct 30 22:28:56.153491 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.153385    4536 scope.go:122] "RemoveContainer" containerID="c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"
Oct 30 22:28:56.163326 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.163300    4536 scope.go:122] "RemoveContainer" containerID="d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430"
Oct 30 22:28:56.163686 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:56.163664    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430\": container with ID starting with d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430 not found: ID does not exist" containerID="d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430"
Oct 30 22:28:56.163800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.163701    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430"} err="failed to get container status \"d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430\": rpc error: code = NotFound desc = could not find container \"d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430\": container with ID starting with d590089aa07f430fb4e8c09dd0fd5e09b15d40e5f85603adbee6a2bb318ec430 not found: ID does not exist"
Oct 30 22:28:56.163800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.163725    4536 scope.go:122] "RemoveContainer" containerID="c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"
Oct 30 22:28:56.163976 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:28:56.163958    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0\": container with ID starting with c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0 not found: ID does not exist" containerID="c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"
Oct 30 22:28:56.164021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.163983    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0"} err="failed to get container status \"c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0\": rpc error: code = NotFound desc = could not find container \"c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0\": container with ID starting with c5acb00a6ed0cfe16ce2aec255961bda010a8c62d2226b871761a27cecd44bd0 not found: ID does not exist"
Oct 30 22:28:56.313155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.313121    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"]
Oct 30 22:28:56.317988 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:56.317954    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2dd57a-openshift-default-5b7c958685-mnsgk"]
Oct 30 22:28:57.216199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:28:57.214880    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe027fb4-c600-4b5d-804e-966790379969" path="/var/lib/kubelet/pods/fe027fb4-c600-4b5d-804e-966790379969/volumes"
Oct 30 22:29:12.255681 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.255640    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"]
Oct 30 22:29:12.256283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.255912    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" containerID="cri-o://7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09" gracePeriod=30
Oct 30 22:29:12.306512 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.306475    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"]
Oct 30 22:29:12.306800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.306748    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy" containerID="cri-o://d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf" gracePeriod=30
Oct 30 22:29:12.310583 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.310559    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"]
Oct 30 22:29:12.310824 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.310779    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy" containerID="cri-o://17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770" gracePeriod=30
Oct 30 22:29:12.356100 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.356070    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"]
Oct 30 22:29:12.356284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:12.356263    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy" containerID="cri-o://cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716" gracePeriod=30
Oct 30 22:29:16.785600 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:16.785559    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mjr2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:29:16.786088 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:16.785640    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:29:17.596101 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.596078    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:29:17.616568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.616523    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.616736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.616605    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.616736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.616665    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.616736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.616710    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.616905 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.616861    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.616987 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.616965    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.617043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.617031    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.617123 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.617109    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.617438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.617233    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy\") pod \"c2e5454b-4393-4649-aeb0-a58220823698\" (UID: \"c2e5454b-4393-4649-aeb0-a58220823698\") "
Oct 30 22:29:17.617774 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.617599    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.618086 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.618057    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.618196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.618157    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.618196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.618171    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.618313 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.618263    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:17.618369 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.618343    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.622585 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.622443    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:17.623940 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.623916    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.629900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.629877    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "kube-api-access-k9rkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.630188 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.630168    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token" pod "c2e5454b-4393-4649-aeb0-a58220823698" (UID: "c2e5454b-4393-4649-aeb0-a58220823698"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.689621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.689585    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:29:17.701153 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.701130    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:29:17.718505 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718477    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.718650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718537    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.718650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718569    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.718650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718592    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.718821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718770    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.718821 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718811    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.718931 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718881    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.718985 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.718972    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.719046 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719010    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.719102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719044    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.719102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719073    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.719214 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719103    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.719214 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719187    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719345    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719369    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719373    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719485    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719524    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719640    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719666    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719676    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket\") pod \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\" (UID: \"d55b555e-0b69-4143-8ed3-95ed8cf39b9d\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719744    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719743    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719778    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token\") pod \"c4148b30-7fb7-43fe-83da-d0b0169d2284\" (UID: \"c4148b30-7fb7-43fe-83da-d0b0169d2284\") "
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719893    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.719963    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:17.720213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720076    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720372    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720496    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720520    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4148b30-7fb7-43fe-83da-d0b0169d2284-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720545    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720560    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720576    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720592    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9rkp\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-kube-api-access-k9rkp\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720608    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720622    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720636    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720651    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720663    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e5454b-4393-4649-aeb0-a58220823698-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720676    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720689    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e5454b-4393-4649-aeb0-a58220823698-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720704    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e5454b-4393-4649-aeb0-a58220823698-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720720    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720734    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e5454b-4393-4649-aeb0-a58220823698-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.720980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.720749    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.721835 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.721331    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "kube-api-access-2pnwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.721835 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.721372    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.722460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.722432    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:17.722814 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.722792    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.722916 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.722890    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "kube-api-access-2jhhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.723051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.722998    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy" pod "c4148b30-7fb7-43fe-83da-d0b0169d2284" (UID: "c4148b30-7fb7-43fe-83da-d0b0169d2284"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.723681 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.723664    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:17.724287 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.724273    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy" pod "d55b555e-0b69-4143-8ed3-95ed8cf39b9d" (UID: "d55b555e-0b69-4143-8ed3-95ed8cf39b9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.725900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.725884    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:29:17.822026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.821765    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.821815    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.821857    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.821882    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.821919    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822002    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822037    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822308    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822364    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs\") pod \"40f74cfd-0389-41ff-88b5-80e17360fcb8\" (UID: \"40f74cfd-0389-41ff-88b5-80e17360fcb8\") "
Oct 30 22:29:17.822765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822428    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.822765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822494    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:17.822765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822739    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.823072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.822916    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.823219 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823196    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823227    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823243    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823256    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823269    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pnwb\" (UniqueName: \"kubernetes.io/projected/c4148b30-7fb7-43fe-83da-d0b0169d2284-kube-api-access-2pnwb\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823284    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jhhm\" (UniqueName: \"kubernetes.io/projected/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-kube-api-access-2jhhm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823298    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823303    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823324    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823342    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4148b30-7fb7-43fe-83da-d0b0169d2284-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823362    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823378    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55b555e-0b69-4143-8ed3-95ed8cf39b9d-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823390    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.823495 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.823423    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f74cfd-0389-41ff-88b5-80e17360fcb8-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.825607 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.825575    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "kube-api-access-r7xrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.825722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.825679    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.825845 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.825825    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:17.826344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.826322    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo" pod "40f74cfd-0389-41ff-88b5-80e17360fcb8" (UID: "40f74cfd-0389-41ff-88b5-80e17360fcb8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:17.925146 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.925105    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7xrx\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-kube-api-access-r7xrx\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.925146 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.925138    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.925146 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.925149    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.925483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.925158    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.925483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:17.925166    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f74cfd-0389-41ff-88b5-80e17360fcb8-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:18.356741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.356707    4536 generic.go:356] "Generic (PLEG): container finished" podID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerID="d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf" exitCode=0
Oct 30 22:29:18.356929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.356760    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"
Oct 30 22:29:18.356929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.356796    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" event={"ID":"d55b555e-0b69-4143-8ed3-95ed8cf39b9d","Type":"ContainerDied","Data":"d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf"}
Oct 30 22:29:18.356929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.356849    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2" event={"ID":"d55b555e-0b69-4143-8ed3-95ed8cf39b9d","Type":"ContainerDied","Data":"ae678f7d54a55562db3af4d49592017d475baef0eecf1dab388c8f2c1a4f1654"}
Oct 30 22:29:18.356929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.356872    4536 scope.go:122] "RemoveContainer" containerID="d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf"
Oct 30 22:29:18.358424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.358369    4536 generic.go:356] "Generic (PLEG): container finished" podID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerID="17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770" exitCode=0
Oct 30 22:29:18.358531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.358457    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"
Oct 30 22:29:18.358531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.358446    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" event={"ID":"c4148b30-7fb7-43fe-83da-d0b0169d2284","Type":"ContainerDied","Data":"17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770"}
Oct 30 22:29:18.358531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.358508    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g" event={"ID":"c4148b30-7fb7-43fe-83da-d0b0169d2284","Type":"ContainerDied","Data":"7a02c33932910205c7ea73f04f81f313d23c3c77d8d793d1241cfd897c27e2bb"}
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.360355    4536 generic.go:356] "Generic (PLEG): container finished" podID="c2e5454b-4393-4649-aeb0-a58220823698" containerID="7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09" exitCode=0
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.360431    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.360460    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" event={"ID":"c2e5454b-4393-4649-aeb0-a58220823698","Type":"ContainerDied","Data":"7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09"}
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.360499    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j" event={"ID":"c2e5454b-4393-4649-aeb0-a58220823698","Type":"ContainerDied","Data":"9d5b9cee719f900151916eea1c9bc92d9bf9709bc4d38ec29111af3d6e87e817"}
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.361992    4536 generic.go:356] "Generic (PLEG): container finished" podID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerID="cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716" exitCode=0
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.362030    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" event={"ID":"40f74cfd-0389-41ff-88b5-80e17360fcb8","Type":"ContainerDied","Data":"cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716"}
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.362043    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"
Oct 30 22:29:18.363460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.362065    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr" event={"ID":"40f74cfd-0389-41ff-88b5-80e17360fcb8","Type":"ContainerDied","Data":"987031b20abb4be0bd336e25c7ebae78e40edb6e753de5ee74c5d09b247d5366"}
Oct 30 22:29:18.370994 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.370952    4536 scope.go:122] "RemoveContainer" containerID="d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf"
Oct 30 22:29:18.371306 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:18.371284    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf\": container with ID starting with d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf not found: ID does not exist" containerID="d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf"
Oct 30 22:29:18.371376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.371314    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf"} err="failed to get container status \"d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf\": rpc error: code = NotFound desc = could not find container \"d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf\": container with ID starting with d0454c70fffb31d0c32053a89b44acf74e8f8b0d0830ac765dfa5ea96b7030cf not found: ID does not exist"
Oct 30 22:29:18.371376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.371336    4536 scope.go:122] "RemoveContainer" containerID="17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770"
Oct 30 22:29:18.381464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.381446    4536 scope.go:122] "RemoveContainer" containerID="17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770"
Oct 30 22:29:18.381716 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:18.381698    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770\": container with ID starting with 17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770 not found: ID does not exist" containerID="17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770"
Oct 30 22:29:18.381793 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.381725    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770"} err="failed to get container status \"17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770\": rpc error: code = NotFound desc = could not find container \"17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770\": container with ID starting with 17b7d2004bc3387953ba4be51ee50b317a363022e5b36c7d1e9a0a57245f3770 not found: ID does not exist"
Oct 30 22:29:18.381793 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.381747    4536 scope.go:122] "RemoveContainer" containerID="7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09"
Oct 30 22:29:18.390238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.390218    4536 scope.go:122] "RemoveContainer" containerID="7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09"
Oct 30 22:29:18.390526 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:18.390501    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09\": container with ID starting with 7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09 not found: ID does not exist" containerID="7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09"
Oct 30 22:29:18.390610 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.390537    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09"} err="failed to get container status \"7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09\": rpc error: code = NotFound desc = could not find container \"7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09\": container with ID starting with 7a630f8bfcccb4f76bdc0099fcbbd8f1c7795f1212c0930ef844ec16534dea09 not found: ID does not exist"
Oct 30 22:29:18.390610 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.390560    4536 scope.go:122] "RemoveContainer" containerID="cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716"
Oct 30 22:29:18.399650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.399631    4536 scope.go:122] "RemoveContainer" containerID="cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716"
Oct 30 22:29:18.399922 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:18.399905    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716\": container with ID starting with cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716 not found: ID does not exist" containerID="cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716"
Oct 30 22:29:18.399969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.399928    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716"} err="failed to get container status \"cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716\": rpc error: code = NotFound desc = could not find container \"cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716\": container with ID starting with cfedf8dc61e9e2d3b981ce16ae95daa4aec363d0f785685c60ab9e2c3b1bf716 not found: ID does not exist"
Oct 30 22:29:18.525220 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.525186    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"]
Oct 30 22:29:18.531270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.531244    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mjr2j"]
Oct 30 22:29:18.642784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.642707    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"]
Oct 30 22:29:18.652243 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.652211    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-qmc8g"]
Oct 30 22:29:18.762931 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.762901    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"]
Oct 30 22:29:18.771846 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.771822    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-p5qh2"]
Oct 30 22:29:18.887005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.886966    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"]
Oct 30 22:29:18.907040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:18.906961    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-27rdr"]
Oct 30 22:29:19.166936 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:19.166855    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" path="/var/lib/kubelet/pods/40f74cfd-0389-41ff-88b5-80e17360fcb8/volumes"
Oct 30 22:29:19.167370 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:19.167357    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2e5454b-4393-4649-aeb0-a58220823698" path="/var/lib/kubelet/pods/c2e5454b-4393-4649-aeb0-a58220823698/volumes"
Oct 30 22:29:19.167843 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:19.167829    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" path="/var/lib/kubelet/pods/c4148b30-7fb7-43fe-83da-d0b0169d2284/volumes"
Oct 30 22:29:19.168594 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:19.168579    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" path="/var/lib/kubelet/pods/d55b555e-0b69-4143-8ed3-95ed8cf39b9d/volumes"
Oct 30 22:29:20.806070 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:20.806036    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:29:21.537021 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:21.536987    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"]
Oct 30 22:29:21.537288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:21.537258    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" containerID="cri-o://4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb" gracePeriod=30
Oct 30 22:29:23.545717 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:23.545681    4536 patch_prober.go:29] interesting pod/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:29:23.546100 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:23.545774    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:29:26.808113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.808047    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:29:26.940784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.940749    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.940979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.940808    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.940979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.940838    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.940979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.940877    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.941156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941032    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.941156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941055    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:26.941156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941137    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.941156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941138    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:26.941376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941154    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:26.941376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941165    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.941376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941264    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.941376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941302    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token\") pod \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\" (UID: \"f5ecab24-c840-4db1-9253-f3c9aa15ab0c\") "
Oct 30 22:29:26.941376 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941360    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:26.941628 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941550    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:26.942015 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.941991    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:26.942301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.942020    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:26.942301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.942035    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:26.942301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.942050    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:26.942301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.942104    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:26.943804 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.943775    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "kube-api-access-pr8ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:26.943804 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.943792    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:26.943804 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.943798    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:26.944121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:26.944103    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo" pod "f5ecab24-c840-4db1-9253-f3c9aa15ab0c" (UID: "f5ecab24-c840-4db1-9253-f3c9aa15ab0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:27.043302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.043267    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr8ww\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-kube-api-access-pr8ww\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:27.043302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.043298    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:27.043302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.043309    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:27.043560 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.043317    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ecab24-c840-4db1-9253-f3c9aa15ab0c-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:27.424734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.424644    4536 generic.go:356] "Generic (PLEG): container finished" podID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerID="4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb" exitCode=0
Oct 30 22:29:27.424734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.424709    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"
Oct 30 22:29:27.424734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.424722    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" event={"ID":"f5ecab24-c840-4db1-9253-f3c9aa15ab0c","Type":"ContainerDied","Data":"4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb"}
Oct 30 22:29:27.424979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.424779    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45" event={"ID":"f5ecab24-c840-4db1-9253-f3c9aa15ab0c","Type":"ContainerDied","Data":"5991fda51d3d4964ab124f31e7ad33744289ca1d0faee4e5a237ebde7f32f61f"}
Oct 30 22:29:27.424979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.424809    4536 scope.go:122] "RemoveContainer" containerID="4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb"
Oct 30 22:29:27.437101 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.437085    4536 scope.go:122] "RemoveContainer" containerID="4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb"
Oct 30 22:29:27.437421 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:27.437386    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb\": container with ID starting with 4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb not found: ID does not exist" containerID="4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb"
Oct 30 22:29:27.437481 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.437430    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb"} err="failed to get container status \"4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb\": rpc error: code = NotFound desc = could not find container \"4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb\": container with ID starting with 4dc28c6a279f29030369f1476e24d609e987192ba7fdc1a1d72e61ec20e87eeb not found: ID does not exist"
Oct 30 22:29:27.605601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.605556    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"]
Oct 30 22:29:27.608452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.608423    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"]
Oct 30 22:29:27.608756 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.608722    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" podUID="54b62d3f-56ba-4357-8776-e8cafb8e3a3f" containerName="maas-api" containerID="cri-o://f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba" gracePeriod=30
Oct 30 22:29:27.645634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:27.645599    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-f9f45"]
Oct 30 22:29:28.108644 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.105465    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:29:28.195623 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.195586    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf\" (UniqueName: \"kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf\") pod \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") "
Oct 30 22:29:28.195796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.195657    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs\") pod \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") "
Oct 30 22:29:28.195892 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.195877    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\") pod \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") "
Oct 30 22:29:28.195967 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.195922    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca\") pod \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\" (UID: \"54b62d3f-56ba-4357-8776-e8cafb8e3a3f\") "
Oct 30 22:29:28.196380 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.196346    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca" pod "54b62d3f-56ba-4357-8776-e8cafb8e3a3f" (UID: "54b62d3f-56ba-4357-8776-e8cafb8e3a3f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:28.196823 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.196802    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.197993 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.197965    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs" pod "54b62d3f-56ba-4357-8776-e8cafb8e3a3f" (UID: "54b62d3f-56ba-4357-8776-e8cafb8e3a3f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:28.198073 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.198022    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf" pod "54b62d3f-56ba-4357-8776-e8cafb8e3a3f" (UID: "54b62d3f-56ba-4357-8776-e8cafb8e3a3f"). InnerVolumeSpecName "kube-api-access-hpgpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:28.198532 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.198517    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls" pod "54b62d3f-56ba-4357-8776-e8cafb8e3a3f" (UID: "54b62d3f-56ba-4357-8776-e8cafb8e3a3f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:28.298662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.298579    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.298662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.298612    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpgpf\" (UniqueName: \"kubernetes.io/projected/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-kube-api-access-hpgpf\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.298662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.298628    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54b62d3f-56ba-4357-8776-e8cafb8e3a3f-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.433470 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.433432    4536 generic.go:356] "Generic (PLEG): container finished" podID="54b62d3f-56ba-4357-8776-e8cafb8e3a3f" containerID="f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba" exitCode=0
Oct 30 22:29:28.433663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.433526    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"
Oct 30 22:29:28.433663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.433567    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" event={"ID":"54b62d3f-56ba-4357-8776-e8cafb8e3a3f","Type":"ContainerDied","Data":"f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba"}
Oct 30 22:29:28.433663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.433627    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7" event={"ID":"54b62d3f-56ba-4357-8776-e8cafb8e3a3f","Type":"ContainerDied","Data":"6bd341b90b6651f368ddfa34c4be4edbaf6731632581d2dc91624bccad116db5"}
Oct 30 22:29:28.433663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.433655    4536 scope.go:122] "RemoveContainer" containerID="f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba"
Oct 30 22:29:28.443420 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.443381    4536 scope.go:122] "RemoveContainer" containerID="f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba"
Oct 30 22:29:28.443711 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:28.443696    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba\": container with ID starting with f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba not found: ID does not exist" containerID="f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba"
Oct 30 22:29:28.443754 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.443719    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba"} err="failed to get container status \"f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba\": rpc error: code = NotFound desc = could not find container \"f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba\": container with ID starting with f2e55681742c7fcf41811cb639d24628d4c01d8d74655bb968eabc7591a72eba not found: ID does not exist"
Oct 30 22:29:28.644430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.641480    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"]
Oct 30 22:29:28.644430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:28.644242    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a42692-66ff96bc95-vgxm7"]
Oct 30 22:29:29.165838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:29.165809    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54b62d3f-56ba-4357-8776-e8cafb8e3a3f" path="/var/lib/kubelet/pods/54b62d3f-56ba-4357-8776-e8cafb8e3a3f/volumes"
Oct 30 22:29:29.166290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:29.166276    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" path="/var/lib/kubelet/pods/f5ecab24-c840-4db1-9253-f3c9aa15ab0c/volumes"
Oct 30 22:29:40.475801 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:40.475725    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"]
Oct 30 22:29:40.476226 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:40.475934    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy" containerID="cri-o://3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb" gracePeriod=30
Oct 30 22:29:40.801738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:40.801649    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"]
Oct 30 22:29:40.801918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:40.801887    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy" containerID="cri-o://fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b" gracePeriod=30
Oct 30 22:29:45.626063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.626027    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"]
Oct 30 22:29:45.738390 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.738354    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:29:45.761642 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761566    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.761791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761652    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.761791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761683    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.761791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761777    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.761959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761824    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.761959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761848    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:45.761959 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761855    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.762119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.761955    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.762119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762020    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.762119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762031    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:45.762119 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762089    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:45.762327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762122    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo\") pod \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\" (UID: \"a5b1de6f-91a5-4948-abe0-e96e29cb68e5\") "
Oct 30 22:29:45.762327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762273    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:45.762456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762350    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:45.762672 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762653    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.762734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762677    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.762734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762696    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.762734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762711    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.762734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.762724    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.764360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.764337    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:45.764603 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.764581    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:45.764687 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.764620    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:45.764741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.764726    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp" pod "a5b1de6f-91a5-4948-abe0-e96e29cb68e5" (UID: "a5b1de6f-91a5-4948-abe0-e96e29cb68e5"). InnerVolumeSpecName "kube-api-access-k6tnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:45.863513 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.863475    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.863513 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.863506    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.863687 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.863519    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:45.863687 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:45.863555    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6tnp\" (UniqueName: \"kubernetes.io/projected/a5b1de6f-91a5-4948-abe0-e96e29cb68e5-kube-api-access-k6tnp\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.056344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.056320    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:29:46.166494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166463    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.166660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166514    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.166660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166561    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.166660 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166636    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.166839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166666    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.166839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166722    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:46.166839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166748    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.166839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166816    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.167039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166847    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.167039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166890    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:46.167039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.166957    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy\") pod \"a281ebb8-b31d-48f6-a61e-99c710ebab88\" (UID: \"a281ebb8-b31d-48f6-a61e-99c710ebab88\") "
Oct 30 22:29:46.167187 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167094    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167225    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167526    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167745    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167818    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167878    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a281ebb8-b31d-48f6-a61e-99c710ebab88-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167899    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.168284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.167914    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.169733 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.169711    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "kube-api-access-p5l46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:46.169808 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.169741    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:29:46.169875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.169861    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:29:46.170322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.170305    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token" pod "a281ebb8-b31d-48f6-a61e-99c710ebab88" (UID: "a281ebb8-b31d-48f6-a61e-99c710ebab88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:46.270388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.270357    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.270388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.270384    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5l46\" (UniqueName: \"kubernetes.io/projected/a281ebb8-b31d-48f6-a61e-99c710ebab88-kube-api-access-p5l46\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.270388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.270422    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.270707 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.270432    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a281ebb8-b31d-48f6-a61e-99c710ebab88-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:46.537047 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.537005    4536 generic.go:356] "Generic (PLEG): container finished" podID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerID="3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb" exitCode=0
Oct 30 22:29:46.537201 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.537125    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"
Oct 30 22:29:46.537201 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.537142    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" event={"ID":"a5b1de6f-91a5-4948-abe0-e96e29cb68e5","Type":"ContainerDied","Data":"3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb"}
Oct 30 22:29:46.537315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.537202    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f" event={"ID":"a5b1de6f-91a5-4948-abe0-e96e29cb68e5","Type":"ContainerDied","Data":"b9fd78deef6d88ceba976bf78f582adc596a5f176c8dd6c45b797beaaa29a0a8"}
Oct 30 22:29:46.537315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.537233    4536 scope.go:122] "RemoveContainer" containerID="3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb"
Oct 30 22:29:46.538658 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.538633    4536 generic.go:356] "Generic (PLEG): container finished" podID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerID="fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b" exitCode=0
Oct 30 22:29:46.538768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.538677    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" event={"ID":"a281ebb8-b31d-48f6-a61e-99c710ebab88","Type":"ContainerDied","Data":"fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b"}
Oct 30 22:29:46.538768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.538684    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"
Oct 30 22:29:46.538768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.538707    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5" event={"ID":"a281ebb8-b31d-48f6-a61e-99c710ebab88","Type":"ContainerDied","Data":"3b47c5b6403cc58ed72639ae9b7435a753350b8c66c921f7ea01e91ee248dbcf"}
Oct 30 22:29:46.547598 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.547579    4536 scope.go:122] "RemoveContainer" containerID="3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb"
Oct 30 22:29:46.547899 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:46.547880    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb\": container with ID starting with 3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb not found: ID does not exist" containerID="3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb"
Oct 30 22:29:46.547981 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.547912    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb"} err="failed to get container status \"3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb\": rpc error: code = NotFound desc = could not find container \"3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb\": container with ID starting with 3a698fd7d6355b8c776afbd40e218bf45838795f05dead781498d87c0550a9cb not found: ID does not exist"
Oct 30 22:29:46.547981 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.547937    4536 scope.go:122] "RemoveContainer" containerID="fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b"
Oct 30 22:29:46.557464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.557445    4536 scope.go:122] "RemoveContainer" containerID="fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b"
Oct 30 22:29:46.557741 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:46.557724    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b\": container with ID starting with fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b not found: ID does not exist" containerID="fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b"
Oct 30 22:29:46.557794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.557748    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b"} err="failed to get container status \"fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b\": rpc error: code = NotFound desc = could not find container \"fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b\": container with ID starting with fb3c0356b02d73c10f905fe4fe785cc60e584b55111df994123b1b02b7d65d1b not found: ID does not exist"
Oct 30 22:29:46.706345 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.706308    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"]
Oct 30 22:29:46.711867 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.711843    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-13b8d8-openshift-default-55bd664645-npdw5"]
Oct 30 22:29:46.824587 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.824502    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"]
Oct 30 22:29:46.833238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:46.833203    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-075cbb-openshift-default-655db97969-xsd6f"]
Oct 30 22:29:47.165538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:47.165460    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" path="/var/lib/kubelet/pods/a281ebb8-b31d-48f6-a61e-99c710ebab88/volumes"
Oct 30 22:29:47.165979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:29:47.165965    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" path="/var/lib/kubelet/pods/a5b1de6f-91a5-4948-abe0-e96e29cb68e5/volumes"
Oct 30 22:29:50.805791 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:29:50.805748    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:30:00.231378 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231346    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"]
Oct 30 22:30:00.231883 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231868    4536 cpu_manager.go:414] "containerMap: removing container" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy"
Oct 30 22:30:00.231928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231884    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy"
Oct 30 22:30:00.231928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231900    4536 cpu_manager.go:414] "containerMap: removing container" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy"
Oct 30 22:30:00.231928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231905    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy"
Oct 30 22:30:00.231928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231925    4536 cpu_manager.go:414] "containerMap: removing container" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231930    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231940    4536 cpu_manager.go:414] "containerMap: removing container" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231944    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231953    4536 cpu_manager.go:414] "containerMap: removing container" podUID="54b62d3f-56ba-4357-8776-e8cafb8e3a3f" containerName="maas-api"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231957    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54b62d3f-56ba-4357-8776-e8cafb8e3a3f" containerName="maas-api"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231970    4536 cpu_manager.go:414] "containerMap: removing container" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231974    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231982    4536 cpu_manager.go:414] "containerMap: removing container" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231986    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231993    4536 cpu_manager.go:414] "containerMap: removing container" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.231997    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232008    4536 cpu_manager.go:414] "containerMap: removing container" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232015    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232028    4536 cpu_manager.go:414] "containerMap: removing container" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy"
Oct 30 22:30:00.232053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232034    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232115    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="a281ebb8-b31d-48f6-a61e-99c710ebab88" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232123    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232129    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="54b62d3f-56ba-4357-8776-e8cafb8e3a3f" containerName="maas-api"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232135    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4148b30-7fb7-43fe-83da-d0b0169d2284" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232140    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe027fb4-c600-4b5d-804e-966790379969" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232147    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5ecab24-c840-4db1-9253-f3c9aa15ab0c" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232156    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2e5454b-4393-4649-aeb0-a58220823698" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232161    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="d55b555e-0b69-4143-8ed3-95ed8cf39b9d" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232167    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5b1de6f-91a5-4948-abe0-e96e29cb68e5" containerName="istio-proxy"
Oct 30 22:30:00.232551 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.232173    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="40f74cfd-0389-41ff-88b5-80e17360fcb8" containerName="istio-proxy"
Oct 30 22:30:00.235568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.235552    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.283156 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.283104    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"]
Oct 30 22:30:00.291800 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.291764    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"]
Oct 30 22:30:00.297316 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.295704    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.299801 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.299771    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlqnr\" (UniqueName: \"kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.300031 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.300013    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.301160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.301137    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.334057 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.334025    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"]
Oct 30 22:30:00.337929 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.337904    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"]
Oct 30 22:30:00.342256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.342238    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.347746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.347710    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"]
Oct 30 22:30:00.350033 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.348241    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-l4mjj\""
Oct 30 22:30:00.383600 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.383570    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"]
Oct 30 22:30:00.391802 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.391773    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"]
Oct 30 22:30:00.401062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401035    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.401213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401064    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5pjj\" (UniqueName: \"kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.401213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401101    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.401213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401170    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.401410 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401337    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlqnr\" (UniqueName: \"kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.401410 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401379    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqm2x\" (UniqueName: \"kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.401535 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401471    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.401659 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401638    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdl6f\" (UniqueName: \"kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.401802 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.401786    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.463825 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.463789    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlqnr\" (UniqueName: \"kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.502553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.502480    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdl6f\" (UniqueName: \"kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.502553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.502533    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.502731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.502554    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5pjj\" (UniqueName: \"kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.502731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.502603    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.502731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.502690    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqm2x\" (UniqueName: \"kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.502872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.502748    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.503283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.503252    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.503416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.503374    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.503461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.503376    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.548200 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.548171    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:00.565432 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.565375    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5pjj\" (UniqueName: \"kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.565540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.565454    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqm2x\" (UniqueName: \"kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.565902 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.565883    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdl6f\" (UniqueName: \"kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f\") pod \"maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.623659 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.623628    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:00.630611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.630578    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:00.649210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.649168    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" containerID="cri-o://b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15" gracePeriod=45
Oct 30 22:30:00.666197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.666108    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:00.687835 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.687791    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 30 22:30:00.950581 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.950548    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"]
Oct 30 22:30:00.976674 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:00.976647    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:30:01.012984 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.012944    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm\") pod \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") "
Oct 30 22:30:01.013180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.013063    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir\") pod \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") "
Oct 30 22:30:01.013180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.013105    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache\") pod \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") "
Oct 30 22:30:01.013180 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.013157    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home\") pod \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") "
Oct 30 22:30:01.013374 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.013203    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs\") pod \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") "
Oct 30 22:30:01.013374 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.013261    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz\" (UniqueName: \"kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz\") pod \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\" (UID: \"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f\") "
Oct 30 22:30:01.013709 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.013657    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir" pod "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" (UID: "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.014049 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.014020    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home" pod "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" (UID: "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.014826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.014799    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache" pod "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" (UID: "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.016327 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.016290    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm" pod "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" (UID: "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:01.016580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.016379    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz" pod "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" (UID: "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f"). InnerVolumeSpecName "kube-api-access-n7dwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:01.016705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.016686    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs" pod "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" (UID: "2f1e0b03-63d1-4636-ab80-c8298ebe8a6f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:01.047997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.047968    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"]
Oct 30 22:30:01.049812 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:30:01.049783    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843976e7_0a3e_4fe5_b192_8a16dcc316c1.slice/crio-4ce145d7ad6f391212c6d8d2083d3a74068da17b4bd7cf10dec010b9ad96178c WatchSource:0}: Error finding container 4ce145d7ad6f391212c6d8d2083d3a74068da17b4bd7cf10dec010b9ad96178c: Status 404 returned error can't find the container with id 4ce145d7ad6f391212c6d8d2083d3a74068da17b4bd7cf10dec010b9ad96178c
Oct 30 22:30:01.067193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.067167    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"]
Oct 30 22:30:01.068794 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:30:01.068770    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb3e529_4595_4efd_8623_eb810c224b99.slice/crio-f56971dac9bae480b363833913b42747a74e7b56ec7f10d348e215abcfef0921 WatchSource:0}: Error finding container f56971dac9bae480b363833913b42747a74e7b56ec7f10d348e215abcfef0921: Status 404 returned error can't find the container with id f56971dac9bae480b363833913b42747a74e7b56ec7f10d348e215abcfef0921
Oct 30 22:30:01.071141 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.071112    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"]
Oct 30 22:30:01.071749 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:30:01.071727    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8dbbc5_31ec_480e_99f5_9aaba2eac22e.slice/crio-dc1803f8ddca109393f87e753e3ada285bad993667e85301e19b6ce80f094641 WatchSource:0}: Error finding container dc1803f8ddca109393f87e753e3ada285bad993667e85301e19b6ce80f094641: Status 404 returned error can't find the container with id dc1803f8ddca109393f87e753e3ada285bad993667e85301e19b6ce80f094641
Oct 30 22:30:01.116228 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.116198    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.116517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.116492    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.116517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.116519    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.116723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.116542    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.116723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.116555    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7dwz\" (UniqueName: \"kubernetes.io/projected/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-kube-api-access-n7dwz\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.116723 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.116569    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:01.635543 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.635502    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerStarted","Data":"4ce145d7ad6f391212c6d8d2083d3a74068da17b4bd7cf10dec010b9ad96178c"}
Oct 30 22:30:01.636953 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.636920    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerStarted","Data":"32aac83720c6c9481b3a7fcaad1412c6abc4edf1021a180ddd120543713d871f"}
Oct 30 22:30:01.638410 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.638375    4536 generic.go:356] "Generic (PLEG): container finished" podID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerID="b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15" exitCode=0
Oct 30 22:30:01.638503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.638429    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" event={"ID":"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f","Type":"ContainerDied","Data":"b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15"}
Oct 30 22:30:01.638503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.638457    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"
Oct 30 22:30:01.638503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.638474    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6" event={"ID":"2f1e0b03-63d1-4636-ab80-c8298ebe8a6f","Type":"ContainerDied","Data":"b89ead380df8757e494bc0e37257d478287b94a74ee95e1513bffb0d0aff6635"}
Oct 30 22:30:01.638503 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.638497    4536 scope.go:122] "RemoveContainer" containerID="b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15"
Oct 30 22:30:01.640168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.640142    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerStarted","Data":"f56971dac9bae480b363833913b42747a74e7b56ec7f10d348e215abcfef0921"}
Oct 30 22:30:01.641742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.641724    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerStarted","Data":"dc1803f8ddca109393f87e753e3ada285bad993667e85301e19b6ce80f094641"}
Oct 30 22:30:01.647918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.647898    4536 scope.go:122] "RemoveContainer" containerID="b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15"
Oct 30 22:30:01.648177 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:01.648160    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15\": container with ID starting with b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15 not found: ID does not exist" containerID="b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15"
Oct 30 22:30:01.648229 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.648188    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15"} err="failed to get container status \"b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15\": rpc error: code = NotFound desc = could not find container \"b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15\": container with ID starting with b5d7c1365a1444e8da9f9af8cbe9ee53ec80a7f92d9f343aaa0c087663ca4c15 not found: ID does not exist"
Oct 30 22:30:01.811210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.811130    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"]
Oct 30 22:30:01.816506 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:01.816470    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-963c3a/distinct-w3-963c3a-kserve-79bd676699-gf5h6"]
Oct 30 22:30:03.190736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:03.190635    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" path="/var/lib/kubelet/pods/2f1e0b03-63d1-4636-ab80-c8298ebe8a6f/volumes"
Oct 30 22:30:06.684762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.684726    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerStarted","Data":"5a163ffdca3428dc0910d44f7b24788f8f9a16282f2368e83cbf4e0282d4fb16"}
Oct 30 22:30:06.686247 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.686221    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerStarted","Data":"0501e91c9a5e4a514756927e89242bbd27aed661bc68ecd4a43129aafc02bafd"}
Oct 30 22:30:06.687694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.687671    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerStarted","Data":"5897afed4f9e9f05e907c957728c2f0cb4da2bfb3a1d4ae1b0efe01869dde108"}
Oct 30 22:30:06.689149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.689126    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerStarted","Data":"ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"}
Oct 30 22:30:06.689306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.689292    4536 scope.go:122] "RemoveContainer" containerID="ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"
Oct 30 22:30:06.803899 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.803844    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" podStartSLOduration=2.054958381 podStartE2EDuration="6.803829267s" totalImagesPullingTime="4.748870886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:06.801908511 +0000 UTC m=+2137.789716973" watchObservedRunningTime="2026-10-30 22:30:06.803829267 +0000 UTC m=+2137.791637587"
Oct 30 22:30:06.920745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:06.920682    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" podStartSLOduration=1.9928669060000002 podStartE2EDuration="6.920666436s" totalImagesPullingTime="4.92779953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:06.918297723 +0000 UTC m=+2137.906105986" watchObservedRunningTime="2026-10-30 22:30:06.920666436 +0000 UTC m=+2137.908474711"
Oct 30 22:30:07.155592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.155481    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" podStartSLOduration=2.429083962 podStartE2EDuration="7.155465103s" totalImagesPullingTime="4.726381141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:07.037733017 +0000 UTC m=+2138.025541311" watchObservedRunningTime="2026-10-30 22:30:07.155465103 +0000 UTC m=+2138.143273407"
Oct 30 22:30:07.696524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.696499    4536 generic.go:356] "Generic (PLEG): container finished" podID="4cb3e529-4595-4efd-8623-eb810c224b99" containerID="5a163ffdca3428dc0910d44f7b24788f8f9a16282f2368e83cbf4e0282d4fb16" exitCode=60
Oct 30 22:30:07.696877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.696586    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerDied","Data":"5a163ffdca3428dc0910d44f7b24788f8f9a16282f2368e83cbf4e0282d4fb16"}
Oct 30 22:30:07.696877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.696832    4536 scope.go:122] "RemoveContainer" containerID="5a163ffdca3428dc0910d44f7b24788f8f9a16282f2368e83cbf4e0282d4fb16"
Oct 30 22:30:07.698328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.698305    4536 generic.go:356] "Generic (PLEG): container finished" podID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerID="0501e91c9a5e4a514756927e89242bbd27aed661bc68ecd4a43129aafc02bafd" exitCode=60
Oct 30 22:30:07.698452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.698430    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerDied","Data":"0501e91c9a5e4a514756927e89242bbd27aed661bc68ecd4a43129aafc02bafd"}
Oct 30 22:30:07.698761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.698745    4536 scope.go:122] "RemoveContainer" containerID="0501e91c9a5e4a514756927e89242bbd27aed661bc68ecd4a43129aafc02bafd"
Oct 30 22:30:07.700054 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.700033    4536 generic.go:356] "Generic (PLEG): container finished" podID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerID="5897afed4f9e9f05e907c957728c2f0cb4da2bfb3a1d4ae1b0efe01869dde108" exitCode=60
Oct 30 22:30:07.700170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.700077    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerDied","Data":"5897afed4f9e9f05e907c957728c2f0cb4da2bfb3a1d4ae1b0efe01869dde108"}
Oct 30 22:30:07.700494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.700472    4536 scope.go:122] "RemoveContainer" containerID="5897afed4f9e9f05e907c957728c2f0cb4da2bfb3a1d4ae1b0efe01869dde108"
Oct 30 22:30:07.710334 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.709953    4536 generic.go:356] "Generic (PLEG): container finished" podID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerID="ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c" exitCode=60
Oct 30 22:30:07.710334 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.709976    4536 generic.go:356] "Generic (PLEG): container finished" podID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerID="297892e0280e73bc78befd0a09243dbe5a2ecf2db47e3dac27d8c85416b3207d" exitCode=60
Oct 30 22:30:07.710501 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.710467    4536 scope.go:122] "RemoveContainer" containerID="297892e0280e73bc78befd0a09243dbe5a2ecf2db47e3dac27d8c85416b3207d"
Oct 30 22:30:07.710745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.710726    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerDied","Data":"ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"}
Oct 30 22:30:07.710839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.710802    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerDied","Data":"297892e0280e73bc78befd0a09243dbe5a2ecf2db47e3dac27d8c85416b3207d"}
Oct 30 22:30:07.710893 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.710840    4536 scope.go:122] "RemoveContainer" containerID="ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"
Oct 30 22:30:07.711677 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:07.711645    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng_odh-ai-gateway-infra(33db14df-ef7c-4008-a269-c0c36c8ea9af)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af"
Oct 30 22:30:07.754105 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.754077    4536 scope.go:122] "RemoveContainer" containerID="ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"
Oct 30 22:30:07.754492 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:07.754463    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c\": container with ID starting with ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c not found: ID does not exist" containerID="ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"
Oct 30 22:30:07.754572 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:07.754500    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c"} err="failed to get container status \"ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c\": rpc error: code = NotFound desc = could not find container \"ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c\": container with ID starting with ce9e7dae1727de3e0177531c63f5168979abb5a2b9950f99e2aa7875e8721a1c not found: ID does not exist"
Oct 30 22:30:08.743736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.743707    4536 generic.go:356] "Generic (PLEG): container finished" podID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerID="6c2b92dff0dca8587a81bbed42a79edbde0eb2a8630853a0563aaffe6380638c" exitCode=60
Oct 30 22:30:08.744081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.743784    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerDied","Data":"6c2b92dff0dca8587a81bbed42a79edbde0eb2a8630853a0563aaffe6380638c"}
Oct 30 22:30:08.744081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.743828    4536 scope.go:122] "RemoveContainer" containerID="5897afed4f9e9f05e907c957728c2f0cb4da2bfb3a1d4ae1b0efe01869dde108"
Oct 30 22:30:08.744211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.744196    4536 scope.go:122] "RemoveContainer" containerID="6c2b92dff0dca8587a81bbed42a79edbde0eb2a8630853a0563aaffe6380638c"
Oct 30 22:30:08.744539 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:08.744488    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7_odh-ai-gateway-infra(843976e7-0a3e-4fe5-b192-8a16dcc316c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1"
Oct 30 22:30:08.747112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.747092    4536 scope.go:122] "RemoveContainer" containerID="297892e0280e73bc78befd0a09243dbe5a2ecf2db47e3dac27d8c85416b3207d"
Oct 30 22:30:08.747369 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:08.747347    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng_odh-ai-gateway-infra(33db14df-ef7c-4008-a269-c0c36c8ea9af)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af"
Oct 30 22:30:08.749166 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.749140    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerStarted","Data":"b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"}
Oct 30 22:30:08.749465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.749448    4536 scope.go:122] "RemoveContainer" containerID="b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"
Oct 30 22:30:08.749742 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:08.749710    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp_odh-ai-gateway-infra(4cb3e529-4595-4efd-8623-eb810c224b99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" podUID="4cb3e529-4595-4efd-8623-eb810c224b99"
Oct 30 22:30:08.751624 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.751603    4536 generic.go:356] "Generic (PLEG): container finished" podID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerID="31c8f9bb7c00abe7a48ed5fb04237aa074962f538f7afba0c0cd2a7c38e6115c" exitCode=60
Oct 30 22:30:08.751731 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.751635    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerDied","Data":"31c8f9bb7c00abe7a48ed5fb04237aa074962f538f7afba0c0cd2a7c38e6115c"}
Oct 30 22:30:08.751913 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.751899    4536 scope.go:122] "RemoveContainer" containerID="31c8f9bb7c00abe7a48ed5fb04237aa074962f538f7afba0c0cd2a7c38e6115c"
Oct 30 22:30:08.752127 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:08.752110    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx_odh-ai-gateway-infra(aa8dbbc5-31ec-480e-99f5-9aaba2eac22e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e"
Oct 30 22:30:08.835907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:08.835881    4536 scope.go:122] "RemoveContainer" containerID="0501e91c9a5e4a514756927e89242bbd27aed661bc68ecd4a43129aafc02bafd"
Oct 30 22:30:09.758745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:09.758711    4536 generic.go:356] "Generic (PLEG): container finished" podID="4cb3e529-4595-4efd-8623-eb810c224b99" containerID="b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa" exitCode=60
Oct 30 22:30:09.759193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:09.758794    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerDied","Data":"b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"}
Oct 30 22:30:09.759193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:09.758861    4536 scope.go:122] "RemoveContainer" containerID="5a163ffdca3428dc0910d44f7b24788f8f9a16282f2368e83cbf4e0282d4fb16"
Oct 30 22:30:09.759193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:09.759157    4536 scope.go:122] "RemoveContainer" containerID="b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"
Oct 30 22:30:09.759506 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:09.759483    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp_odh-ai-gateway-infra(4cb3e529-4595-4efd-8623-eb810c224b99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" podUID="4cb3e529-4595-4efd-8623-eb810c224b99"
Oct 30 22:30:09.760838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:09.760818    4536 scope.go:122] "RemoveContainer" containerID="31c8f9bb7c00abe7a48ed5fb04237aa074962f538f7afba0c0cd2a7c38e6115c"
Oct 30 22:30:09.761063 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:09.761045    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx_odh-ai-gateway-infra(aa8dbbc5-31ec-480e-99f5-9aaba2eac22e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e"
Oct 30 22:30:09.762489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:09.762471    4536 scope.go:122] "RemoveContainer" containerID="6c2b92dff0dca8587a81bbed42a79edbde0eb2a8630853a0563aaffe6380638c"
Oct 30 22:30:09.762667 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:09.762649    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7_odh-ai-gateway-infra(843976e7-0a3e-4fe5-b192-8a16dcc316c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1"
Oct 30 22:30:10.769638 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:10.769611    4536 scope.go:122] "RemoveContainer" containerID="b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"
Oct 30 22:30:10.770024 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:10.769878    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp_odh-ai-gateway-infra(4cb3e529-4595-4efd-8623-eb810c224b99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" podUID="4cb3e529-4595-4efd-8623-eb810c224b99"
Oct 30 22:30:20.158914 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:20.158859    4536 scope.go:122] "RemoveContainer" containerID="6c2b92dff0dca8587a81bbed42a79edbde0eb2a8630853a0563aaffe6380638c"
Oct 30 22:30:20.159461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:20.159010    4536 scope.go:122] "RemoveContainer" containerID="b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"
Oct 30 22:30:20.159461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:20.159154    4536 scope.go:122] "RemoveContainer" containerID="297892e0280e73bc78befd0a09243dbe5a2ecf2db47e3dac27d8c85416b3207d"
Oct 30 22:30:20.804982 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:20.804894    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:30:20.828588 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:20.828554    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerStarted","Data":"af997b862fcb6c2141580e00119664f27f6437ab4d0ac720f648ce3fea9b690f"}
Oct 30 22:30:20.830447 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:20.830419    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerStarted","Data":"e27c34d18100e48f5924d45c4a9ae0ed85958a810a294f8e08cac2f6c3726849"}
Oct 30 22:30:20.832411 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:20.832380    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerStarted","Data":"4e705af337c3f58e6227c61aeaab1e54b4994b7c3d6dd9d8980eae9550a6b820"}
Oct 30 22:30:21.839742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.839701    4536 generic.go:356] "Generic (PLEG): container finished" podID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerID="e27c34d18100e48f5924d45c4a9ae0ed85958a810a294f8e08cac2f6c3726849" exitCode=60
Oct 30 22:30:21.840306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.839771    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerDied","Data":"e27c34d18100e48f5924d45c4a9ae0ed85958a810a294f8e08cac2f6c3726849"}
Oct 30 22:30:21.840306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.839832    4536 scope.go:122] "RemoveContainer" containerID="6c2b92dff0dca8587a81bbed42a79edbde0eb2a8630853a0563aaffe6380638c"
Oct 30 22:30:21.840306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.840204    4536 scope.go:122] "RemoveContainer" containerID="e27c34d18100e48f5924d45c4a9ae0ed85958a810a294f8e08cac2f6c3726849"
Oct 30 22:30:21.840580 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:21.840516    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7_odh-ai-gateway-infra(843976e7-0a3e-4fe5-b192-8a16dcc316c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1"
Oct 30 22:30:21.841796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.841768    4536 generic.go:356] "Generic (PLEG): container finished" podID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerID="4e705af337c3f58e6227c61aeaab1e54b4994b7c3d6dd9d8980eae9550a6b820" exitCode=60
Oct 30 22:30:21.841882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.841844    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerDied","Data":"4e705af337c3f58e6227c61aeaab1e54b4994b7c3d6dd9d8980eae9550a6b820"}
Oct 30 22:30:21.842191 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.842175    4536 scope.go:122] "RemoveContainer" containerID="4e705af337c3f58e6227c61aeaab1e54b4994b7c3d6dd9d8980eae9550a6b820"
Oct 30 22:30:21.842540 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:21.842518    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng_odh-ai-gateway-infra(33db14df-ef7c-4008-a269-c0c36c8ea9af)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af"
Oct 30 22:30:21.843673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.843656    4536 generic.go:356] "Generic (PLEG): container finished" podID="4cb3e529-4595-4efd-8623-eb810c224b99" containerID="af997b862fcb6c2141580e00119664f27f6437ab4d0ac720f648ce3fea9b690f" exitCode=60
Oct 30 22:30:21.843754 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.843683    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerDied","Data":"af997b862fcb6c2141580e00119664f27f6437ab4d0ac720f648ce3fea9b690f"}
Oct 30 22:30:21.843986 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.843973    4536 scope.go:122] "RemoveContainer" containerID="af997b862fcb6c2141580e00119664f27f6437ab4d0ac720f648ce3fea9b690f"
Oct 30 22:30:21.844199 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:21.844183    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp_odh-ai-gateway-infra(4cb3e529-4595-4efd-8623-eb810c224b99)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" podUID="4cb3e529-4595-4efd-8623-eb810c224b99"
Oct 30 22:30:21.852882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.852864    4536 scope.go:122] "RemoveContainer" containerID="297892e0280e73bc78befd0a09243dbe5a2ecf2db47e3dac27d8c85416b3207d"
Oct 30 22:30:21.865906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.865878    4536 scope.go:122] "RemoveContainer" containerID="b1080161493960643b03e9e81dcd8339258a8d0f7024dc0386ca5a3ea7fab8aa"
Oct 30 22:30:21.960760 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:21.960708    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" podStartSLOduration=17.281534898 podStartE2EDuration="21.960688622s" totalImagesPullingTime="4.679153724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:21.191653565 +0000 UTC m=+2152.179461827" watchObservedRunningTime="2026-10-30 22:30:21.960688622 +0000 UTC m=+2152.948496900"
Oct 30 22:30:22.008106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:22.008077    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"]
Oct 30 22:30:22.094575 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:22.094468    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"]
Oct 30 22:30:22.203199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:22.203170    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"]
Oct 30 22:30:23.025786 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.025682    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:23.042609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.041883    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x\" (UniqueName: \"kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x\") pod \"4cb3e529-4595-4efd-8623-eb810c224b99\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") "
Oct 30 22:30:23.042609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.042028    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca\") pod \"4cb3e529-4595-4efd-8623-eb810c224b99\" (UID: \"4cb3e529-4595-4efd-8623-eb810c224b99\") "
Oct 30 22:30:23.042825 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.042784    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca" pod "4cb3e529-4595-4efd-8623-eb810c224b99" (UID: "4cb3e529-4595-4efd-8623-eb810c224b99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:23.043022 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.043001    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb3e529-4595-4efd-8623-eb810c224b99-maas-api-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:23.045440 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.045419    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x" pod "4cb3e529-4595-4efd-8623-eb810c224b99" (UID: "4cb3e529-4595-4efd-8623-eb810c224b99"). InnerVolumeSpecName "kube-api-access-lqm2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:23.083591 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.083563    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:23.091750 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.091728    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:23.144085 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.143965    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca\") pod \"33db14df-ef7c-4008-a269-c0c36c8ea9af\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") "
Oct 30 22:30:23.144237 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144136    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca\") pod \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") "
Oct 30 22:30:23.144305 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144280    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr\" (UniqueName: \"kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr\") pod \"33db14df-ef7c-4008-a269-c0c36c8ea9af\" (UID: \"33db14df-ef7c-4008-a269-c0c36c8ea9af\") "
Oct 30 22:30:23.144371 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144322    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj\" (UniqueName: \"kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj\") pod \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\" (UID: \"843976e7-0a3e-4fe5-b192-8a16dcc316c1\") "
Oct 30 22:30:23.144477 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144423    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca" pod "33db14df-ef7c-4008-a269-c0c36c8ea9af" (UID: "33db14df-ef7c-4008-a269-c0c36c8ea9af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:23.144884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144847    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33db14df-ef7c-4008-a269-c0c36c8ea9af-maas-api-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:23.145006 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144892    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqm2x\" (UniqueName: \"kubernetes.io/projected/4cb3e529-4595-4efd-8623-eb810c224b99-kube-api-access-lqm2x\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:23.145006 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.144861    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca" pod "843976e7-0a3e-4fe5-b192-8a16dcc316c1" (UID: "843976e7-0a3e-4fe5-b192-8a16dcc316c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:23.147378 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.147352    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj" pod "843976e7-0a3e-4fe5-b192-8a16dcc316c1" (UID: "843976e7-0a3e-4fe5-b192-8a16dcc316c1"). InnerVolumeSpecName "kube-api-access-x5pjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:23.147516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.147491    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr" pod "33db14df-ef7c-4008-a269-c0c36c8ea9af" (UID: "33db14df-ef7c-4008-a269-c0c36c8ea9af"). InnerVolumeSpecName "kube-api-access-jlqnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:23.162751 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.162309    4536 scope.go:122] "RemoveContainer" containerID="31c8f9bb7c00abe7a48ed5fb04237aa074962f538f7afba0c0cd2a7c38e6115c"
Oct 30 22:30:23.247163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.247132    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5pjj\" (UniqueName: \"kubernetes.io/projected/843976e7-0a3e-4fe5-b192-8a16dcc316c1-kube-api-access-x5pjj\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:23.247482 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.247464    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/843976e7-0a3e-4fe5-b192-8a16dcc316c1-maas-api-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:23.247635 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.247589    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlqnr\" (UniqueName: \"kubernetes.io/projected/33db14df-ef7c-4008-a269-c0c36c8ea9af-kube-api-access-jlqnr\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:23.865352 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.865263    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp" event={"ID":"4cb3e529-4595-4efd-8623-eb810c224b99","Type":"ContainerDied","Data":"f56971dac9bae480b363833913b42747a74e7b56ec7f10d348e215abcfef0921"}
Oct 30 22:30:23.865352 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.865330    4536 scope.go:122] "RemoveContainer" containerID="af997b862fcb6c2141580e00119664f27f6437ab4d0ac720f648ce3fea9b690f"
Oct 30 22:30:23.865596 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.865273    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"
Oct 30 22:30:23.867941 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.867914    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerStarted","Data":"439f43ced912432ef5a08e956571645f5f9b927361fb7ae7d0e1f9e433e7ba53"}
Oct 30 22:30:23.870411 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.870371    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"
Oct 30 22:30:23.870411 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.870378    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7" event={"ID":"843976e7-0a3e-4fe5-b192-8a16dcc316c1","Type":"ContainerDied","Data":"4ce145d7ad6f391212c6d8d2083d3a74068da17b4bd7cf10dec010b9ad96178c"}
Oct 30 22:30:23.872634 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.872614    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng" event={"ID":"33db14df-ef7c-4008-a269-c0c36c8ea9af","Type":"ContainerDied","Data":"32aac83720c6c9481b3a7fcaad1412c6abc4edf1021a180ddd120543713d871f"}
Oct 30 22:30:23.872736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.872712    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"
Oct 30 22:30:23.876932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.876914    4536 scope.go:122] "RemoveContainer" containerID="e27c34d18100e48f5924d45c4a9ae0ed85958a810a294f8e08cac2f6c3726849"
Oct 30 22:30:23.887451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:23.887434    4536 scope.go:122] "RemoveContainer" containerID="4e705af337c3f58e6227c61aeaab1e54b4994b7c3d6dd9d8980eae9550a6b820"
Oct 30 22:30:24.149818 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.149733    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"]
Oct 30 22:30:24.157164 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.157133    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-afe802-29889990-56sd7"]
Oct 30 22:30:24.269445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.269381    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"]
Oct 30 22:30:24.273328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.273285    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-7fd7ba-29889990-4qtng"]
Oct 30 22:30:24.382005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.381978    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"]
Oct 30 22:30:24.388280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.388256    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-8e1227-29889990-mqnzp"]
Oct 30 22:30:24.887167 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.887136    4536 generic.go:356] "Generic (PLEG): container finished" podID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerID="439f43ced912432ef5a08e956571645f5f9b927361fb7ae7d0e1f9e433e7ba53" exitCode=60
Oct 30 22:30:24.887355 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.887177    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerDied","Data":"439f43ced912432ef5a08e956571645f5f9b927361fb7ae7d0e1f9e433e7ba53"}
Oct 30 22:30:24.887355 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.887225    4536 scope.go:122] "RemoveContainer" containerID="31c8f9bb7c00abe7a48ed5fb04237aa074962f538f7afba0c0cd2a7c38e6115c"
Oct 30 22:30:24.887784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:24.887693    4536 scope.go:122] "RemoveContainer" containerID="439f43ced912432ef5a08e956571645f5f9b927361fb7ae7d0e1f9e433e7ba53"
Oct 30 22:30:24.888143 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:24.888066    4536 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx_odh-ai-gateway-infra(aa8dbbc5-31ec-480e-99f5-9aaba2eac22e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e"
Oct 30 22:30:25.049609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:25.049577    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"]
Oct 30 22:30:25.166651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:25.166577    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" path="/var/lib/kubelet/pods/33db14df-ef7c-4008-a269-c0c36c8ea9af/volumes"
Oct 30 22:30:25.167013 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:25.166993    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" path="/var/lib/kubelet/pods/4cb3e529-4595-4efd-8623-eb810c224b99/volumes"
Oct 30 22:30:25.167346 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:25.167333    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" path="/var/lib/kubelet/pods/843976e7-0a3e-4fe5-b192-8a16dcc316c1/volumes"
Oct 30 22:30:26.085455 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.085385    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:26.152819 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.152725    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f\" (UniqueName: \"kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f\") pod \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") "
Oct 30 22:30:26.152979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.152898    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca\") pod \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\" (UID: \"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e\") "
Oct 30 22:30:26.153282 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.153255    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca" pod "aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" (UID: "aa8dbbc5-31ec-480e-99f5-9aaba2eac22e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:26.153734 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.153716    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-maas-api-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.155238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.155209    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f" pod "aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" (UID: "aa8dbbc5-31ec-480e-99f5-9aaba2eac22e"). InnerVolumeSpecName "kube-api-access-hdl6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:26.255335 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.255200    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdl6f\" (UniqueName: \"kubernetes.io/projected/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e-kube-api-access-hdl6f\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.910058 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.910032    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"
Oct 30 22:30:26.910244 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.910079    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx" event={"ID":"aa8dbbc5-31ec-480e-99f5-9aaba2eac22e","Type":"ContainerDied","Data":"dc1803f8ddca109393f87e753e3ada285bad993667e85301e19b6ce80f094641"}
Oct 30 22:30:26.910244 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:26.910154    4536 scope.go:122] "RemoveContainer" containerID="439f43ced912432ef5a08e956571645f5f9b927361fb7ae7d0e1f9e433e7ba53"
Oct 30 22:30:27.078005 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:27.077966    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"]
Oct 30 22:30:27.092606 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:27.092582    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w5-53cceb-29889990-rzngx"]
Oct 30 22:30:27.165749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:27.165674    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" path="/var/lib/kubelet/pods/aa8dbbc5-31ec-480e-99f5-9aaba2eac22e/volumes"
Oct 30 22:30:46.759932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:46.759890    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"]
Oct 30 22:30:46.809309 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:30:46.809276    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"]
Oct 30 22:30:50.805624 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:30:50.805583    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:31:01.781338 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:01.781277    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main" containerID="cri-o://4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019" gracePeriod=45
Oct 30 22:31:01.830171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:01.830107    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" containerID="cri-o://4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf" gracePeriod=45
Oct 30 22:31:02.013544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.013220    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": dial tcp 10.133.0.42:8000: connect: connection refused"
Oct 30 22:31:02.114225 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.114200    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:31:02.117097 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.117077    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:31:02.138736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.138707    4536 generic.go:356] "Generic (PLEG): container finished" podID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerID="4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019" exitCode=0
Oct 30 22:31:02.138911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.138751    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"
Oct 30 22:31:02.138911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.138803    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" event={"ID":"f448fdd2-77cc-4c38-ba05-1078286a2a32","Type":"ContainerDied","Data":"4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019"}
Oct 30 22:31:02.138911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.138850    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv" event={"ID":"f448fdd2-77cc-4c38-ba05-1078286a2a32","Type":"ContainerDied","Data":"b5aba5947cadebcbc7148f37212e0402978a047c157f0b4850e97ad9f5ea5907"}
Oct 30 22:31:02.138911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.138867    4536 scope.go:122] "RemoveContainer" containerID="4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019"
Oct 30 22:31:02.139975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.139953    4536 generic.go:356] "Generic (PLEG): container finished" podID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerID="4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf" exitCode=0
Oct 30 22:31:02.140043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.139996    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"
Oct 30 22:31:02.140108 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.140041    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" event={"ID":"081b695c-a7d3-4519-b0af-745bfcfa2ad1","Type":"ContainerDied","Data":"4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf"}
Oct 30 22:31:02.140108 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.140082    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9" event={"ID":"081b695c-a7d3-4519-b0af-745bfcfa2ad1","Type":"ContainerDied","Data":"beb547f771783cf1004df0d76b686aca9a6c44e366b10e2451706a9a3e1242cf"}
Oct 30 22:31:02.148451 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.148433    4536 scope.go:122] "RemoveContainer" containerID="4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019"
Oct 30 22:31:02.148765 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:02.148742    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019\": container with ID starting with 4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019 not found: ID does not exist" containerID="4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019"
Oct 30 22:31:02.148872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.148773    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019"} err="failed to get container status \"4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019\": rpc error: code = NotFound desc = could not find container \"4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019\": container with ID starting with 4bf2f1755f28cc82c310d7bfb127a71778f4226ddeed0e05438dad8c3b76c019 not found: ID does not exist"
Oct 30 22:31:02.148872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.148791    4536 scope.go:122] "RemoveContainer" containerID="4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf"
Oct 30 22:31:02.157378 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.157350    4536 scope.go:122] "RemoveContainer" containerID="4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf"
Oct 30 22:31:02.166342 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:02.158275    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf\": container with ID starting with 4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf not found: ID does not exist" containerID="4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf"
Oct 30 22:31:02.166342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.158333    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf"} err="failed to get container status \"4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf\": rpc error: code = NotFound desc = could not find container \"4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf\": container with ID starting with 4c16187fb330a83c9186ebd864d8612e38578f36111ce09168ac24ff1090edcf not found: ID does not exist"
Oct 30 22:31:02.255489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255454    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache\") pod \"f448fdd2-77cc-4c38-ba05-1078286a2a32\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") "
Oct 30 22:31:02.255668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255533    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm\") pod \"f448fdd2-77cc-4c38-ba05-1078286a2a32\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") "
Oct 30 22:31:02.255668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255571    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home\") pod \"f448fdd2-77cc-4c38-ba05-1078286a2a32\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") "
Oct 30 22:31:02.255668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255609    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home\") pod \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") "
Oct 30 22:31:02.255668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255647    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl\" (UniqueName: \"kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl\") pod \"f448fdd2-77cc-4c38-ba05-1078286a2a32\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") "
Oct 30 22:31:02.255839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255702    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache\") pod \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") "
Oct 30 22:31:02.255839 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255768    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs\" (UniqueName: \"kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs\") pod \"f448fdd2-77cc-4c38-ba05-1078286a2a32\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") "
Oct 30 22:31:02.255948 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.255914    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home" pod "f448fdd2-77cc-4c38-ba05-1078286a2a32" (UID: "f448fdd2-77cc-4c38-ba05-1078286a2a32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.256108 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.256056    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home" pod "081b695c-a7d3-4519-b0af-745bfcfa2ad1" (UID: "081b695c-a7d3-4519-b0af-745bfcfa2ad1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.256185 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.256115    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache" pod "081b695c-a7d3-4519-b0af-745bfcfa2ad1" (UID: "081b695c-a7d3-4519-b0af-745bfcfa2ad1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.256256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.256237    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir\") pod \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") "
Oct 30 22:31:02.256517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.256492    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache" pod "f448fdd2-77cc-4c38-ba05-1078286a2a32" (UID: "f448fdd2-77cc-4c38-ba05-1078286a2a32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.256677 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.256598    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir\") pod \"f448fdd2-77cc-4c38-ba05-1078286a2a32\" (UID: \"f448fdd2-77cc-4c38-ba05-1078286a2a32\") "
Oct 30 22:31:02.257265 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.257244    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs\" (UniqueName: \"kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs\") pod \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") "
Oct 30 22:31:02.257494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.257476    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm\") pod \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") "
Oct 30 22:31:02.257605 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.256500    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir" pod "081b695c-a7d3-4519-b0af-745bfcfa2ad1" (UID: "081b695c-a7d3-4519-b0af-745bfcfa2ad1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.257746 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.257725    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc\" (UniqueName: \"kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc\") pod \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\" (UID: \"081b695c-a7d3-4519-b0af-745bfcfa2ad1\") "
Oct 30 22:31:02.258542 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.258520    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.258676 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.258660    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.258779 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.258767    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.258898 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.258857    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.259006 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.258992    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.259545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.259525    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir" pod "f448fdd2-77cc-4c38-ba05-1078286a2a32" (UID: "f448fdd2-77cc-4c38-ba05-1078286a2a32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.261449 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.261429    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl" pod "f448fdd2-77cc-4c38-ba05-1078286a2a32" (UID: "f448fdd2-77cc-4c38-ba05-1078286a2a32"). InnerVolumeSpecName "kube-api-access-dzsvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:02.261449 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.261433    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs" pod "f448fdd2-77cc-4c38-ba05-1078286a2a32" (UID: "f448fdd2-77cc-4c38-ba05-1078286a2a32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:02.261626 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.261514    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs" pod "081b695c-a7d3-4519-b0af-745bfcfa2ad1" (UID: "081b695c-a7d3-4519-b0af-745bfcfa2ad1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:02.261626 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.261563    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm" pod "f448fdd2-77cc-4c38-ba05-1078286a2a32" (UID: "f448fdd2-77cc-4c38-ba05-1078286a2a32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.261981 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.261952    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc" pod "081b695c-a7d3-4519-b0af-745bfcfa2ad1" (UID: "081b695c-a7d3-4519-b0af-745bfcfa2ad1"). InnerVolumeSpecName "kube-api-access-qhfxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:02.263317 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.263276    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm" pod "081b695c-a7d3-4519-b0af-745bfcfa2ad1" (UID: "081b695c-a7d3-4519-b0af-745bfcfa2ad1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360781    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360808    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzsvl\" (UniqueName: \"kubernetes.io/projected/f448fdd2-77cc-4c38-ba05-1078286a2a32-kube-api-access-dzsvl\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360818    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f448fdd2-77cc-4c38-ba05-1078286a2a32-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360827    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f448fdd2-77cc-4c38-ba05-1078286a2a32-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360835    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/081b695c-a7d3-4519-b0af-745bfcfa2ad1-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360842    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/081b695c-a7d3-4519-b0af-745bfcfa2ad1-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.360852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.360850    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhfxc\" (UniqueName: \"kubernetes.io/projected/081b695c-a7d3-4519-b0af-745bfcfa2ad1-kube-api-access-qhfxc\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:02.565022 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.564990    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"]
Oct 30 22:31:02.569938 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.569906    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/premium-w2-13b8d8-kserve-5fddc4866-xdhhv"]
Oct 30 22:31:02.685278 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.685154    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"]
Oct 30 22:31:02.689273 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:02.689247    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-13b8d8/simulator-w2-13b8d8-kserve-856d4b4bc9-82ld9"]
Oct 30 22:31:03.165578 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:03.165547    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" path="/var/lib/kubelet/pods/081b695c-a7d3-4519-b0af-745bfcfa2ad1/volumes"
Oct 30 22:31:03.166002 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:03.165973    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" path="/var/lib/kubelet/pods/f448fdd2-77cc-4c38-ba05-1078286a2a32/volumes"
Oct 30 22:31:20.806836 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:20.806748    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:31:24.305425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.303079    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"]
Oct 30 22:31:24.305425 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.303338    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" podUID="70861c3f-0093-47d0-8f26-3f7967f40eb5" containerName="maas-api" containerID="cri-o://974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40" gracePeriod=30
Oct 30 22:31:24.732320 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.732292    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:31:24.865256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.857154    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca\") pod \"70861c3f-0093-47d0-8f26-3f7967f40eb5\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") "
Oct 30 22:31:24.865256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.857258    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\") pod \"70861c3f-0093-47d0-8f26-3f7967f40eb5\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") "
Oct 30 22:31:24.865256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.857307    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs\") pod \"70861c3f-0093-47d0-8f26-3f7967f40eb5\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") "
Oct 30 22:31:24.865256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.857429    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26\" (UniqueName: \"kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26\") pod \"70861c3f-0093-47d0-8f26-3f7967f40eb5\" (UID: \"70861c3f-0093-47d0-8f26-3f7967f40eb5\") "
Oct 30 22:31:24.865256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.858939    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca" pod "70861c3f-0093-47d0-8f26-3f7967f40eb5" (UID: "70861c3f-0093-47d0-8f26-3f7967f40eb5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:24.865256 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.864745    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26" pod "70861c3f-0093-47d0-8f26-3f7967f40eb5" (UID: "70861c3f-0093-47d0-8f26-3f7967f40eb5"). InnerVolumeSpecName "kube-api-access-98v26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:24.869944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.869906    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls" pod "70861c3f-0093-47d0-8f26-3f7967f40eb5" (UID: "70861c3f-0093-47d0-8f26-3f7967f40eb5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:24.876140 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.876076    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs" pod "70861c3f-0093-47d0-8f26-3f7967f40eb5" (UID: "70861c3f-0093-47d0-8f26-3f7967f40eb5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:24.959069 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.959037    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98v26\" (UniqueName: \"kubernetes.io/projected/70861c3f-0093-47d0-8f26-3f7967f40eb5-kube-api-access-98v26\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:24.959069 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.959066    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70861c3f-0093-47d0-8f26-3f7967f40eb5-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:24.959069 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.959076    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:24.959304 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:24.959086    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70861c3f-0093-47d0-8f26-3f7967f40eb5-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:25.312152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.310610    4536 generic.go:356] "Generic (PLEG): container finished" podID="70861c3f-0093-47d0-8f26-3f7967f40eb5" containerID="974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40" exitCode=0
Oct 30 22:31:25.312152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.310716    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" event={"ID":"70861c3f-0093-47d0-8f26-3f7967f40eb5","Type":"ContainerDied","Data":"974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40"}
Oct 30 22:31:25.312152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.310770    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc" event={"ID":"70861c3f-0093-47d0-8f26-3f7967f40eb5","Type":"ContainerDied","Data":"4782829ca91ac704d2a5e965b89fbf5b946b018d342f2c21539c6d1051d23484"}
Oct 30 22:31:25.312152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.310955    4536 scope.go:122] "RemoveContainer" containerID="974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40"
Oct 30 22:31:25.312152 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.311229    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"
Oct 30 22:31:25.323975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.323951    4536 scope.go:122] "RemoveContainer" containerID="974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40"
Oct 30 22:31:25.324364 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:25.324330    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40\": container with ID starting with 974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40 not found: ID does not exist" containerID="974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40"
Oct 30 22:31:25.324482 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.324365    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40"} err="failed to get container status \"974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40\": rpc error: code = NotFound desc = could not find container \"974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40\": container with ID starting with 974cd6eccd4717615949477c2fed7429af9433fec17a74fff7eabb5423d8bf40 not found: ID does not exist"
Oct 30 22:31:25.484003 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.483972    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"]
Oct 30 22:31:25.533572 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.533537    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-56c628-5b779b7497-9hdkc"]
Oct 30 22:31:25.726251 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.726206    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"]
Oct 30 22:31:25.734539 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:25.730764    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" podUID="6fed049f-6d70-47db-aeec-2bcbf2271ce9" containerName="maas-api" containerID="cri-o://86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d" gracePeriod=30
Oct 30 22:31:26.090562 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.090537    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:31:26.176918 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.176882    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\") pod \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") "
Oct 30 22:31:26.177102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.176948    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p\" (UniqueName: \"kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p\") pod \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") "
Oct 30 22:31:26.177102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.176994    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca\") pod \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") "
Oct 30 22:31:26.177197 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.177132    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs\") pod \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\" (UID: \"6fed049f-6d70-47db-aeec-2bcbf2271ce9\") "
Oct 30 22:31:26.178662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.178619    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca" pod "6fed049f-6d70-47db-aeec-2bcbf2271ce9" (UID: "6fed049f-6d70-47db-aeec-2bcbf2271ce9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:26.180757 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.180715    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs" pod "6fed049f-6d70-47db-aeec-2bcbf2271ce9" (UID: "6fed049f-6d70-47db-aeec-2bcbf2271ce9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:26.185069 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.185027    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p" pod "6fed049f-6d70-47db-aeec-2bcbf2271ce9" (UID: "6fed049f-6d70-47db-aeec-2bcbf2271ce9"). InnerVolumeSpecName "kube-api-access-zbq5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:26.186099 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.186074    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls" pod "6fed049f-6d70-47db-aeec-2bcbf2271ce9" (UID: "6fed049f-6d70-47db-aeec-2bcbf2271ce9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:26.202183 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.202148    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"]
Oct 30 22:31:26.202434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.202374    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy" containerID="cri-o://8ce213841e3d182c4c21e0e9aed68ef61798fdfe43d41f695554c2a48084d403" gracePeriod=30
Oct 30 22:31:26.278969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.278940    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:26.278969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.278969    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fed049f-6d70-47db-aeec-2bcbf2271ce9-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:26.278969 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.278983    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbq5p\" (UniqueName: \"kubernetes.io/projected/6fed049f-6d70-47db-aeec-2bcbf2271ce9-kube-api-access-zbq5p\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:26.279276 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.278996    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fed049f-6d70-47db-aeec-2bcbf2271ce9-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:26.318798 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.318715    4536 generic.go:356] "Generic (PLEG): container finished" podID="6fed049f-6d70-47db-aeec-2bcbf2271ce9" containerID="86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d" exitCode=0
Oct 30 22:31:26.319249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.318792    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"
Oct 30 22:31:26.319249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.318824    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" event={"ID":"6fed049f-6d70-47db-aeec-2bcbf2271ce9","Type":"ContainerDied","Data":"86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d"}
Oct 30 22:31:26.319249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.318875    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8" event={"ID":"6fed049f-6d70-47db-aeec-2bcbf2271ce9","Type":"ContainerDied","Data":"f0a56edac60a10b39196066d7899a13947c39bf748ac06d074f4fd29a3fb1fdc"}
Oct 30 22:31:26.319249 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.318905    4536 scope.go:122] "RemoveContainer" containerID="86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d"
Oct 30 22:31:26.369625 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.360348    4536 scope.go:122] "RemoveContainer" containerID="86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d"
Oct 30 22:31:26.373418 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:26.371365    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d\": container with ID starting with 86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d not found: ID does not exist" containerID="86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d"
Oct 30 22:31:26.373418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.371437    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d"} err="failed to get container status \"86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d\": rpc error: code = NotFound desc = could not find container \"86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d\": container with ID starting with 86244d0307b0f55c360996a9cb2d02062399b46bb139c0c16fedc4bd76157b2d not found: ID does not exist"
Oct 30 22:31:26.457657 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.457620    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"]
Oct 30 22:31:26.464064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:26.464027    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6d4d3c-869b9d7685-hltx8"]
Oct 30 22:31:27.165962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:27.165925    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fed049f-6d70-47db-aeec-2bcbf2271ce9" path="/var/lib/kubelet/pods/6fed049f-6d70-47db-aeec-2bcbf2271ce9/volumes"
Oct 30 22:31:27.166346 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:27.166331    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70861c3f-0093-47d0-8f26-3f7967f40eb5" path="/var/lib/kubelet/pods/70861c3f-0093-47d0-8f26-3f7967f40eb5/volumes"
Oct 30 22:31:31.362480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.361164    4536 generic.go:356] "Generic (PLEG): container finished" podID="2064d62c-1633-47c4-b455-354994115829" containerID="8ce213841e3d182c4c21e0e9aed68ef61798fdfe43d41f695554c2a48084d403" exitCode=0
Oct 30 22:31:31.362480 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.361272    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" event={"ID":"2064d62c-1633-47c4-b455-354994115829","Type":"ContainerDied","Data":"8ce213841e3d182c4c21e0e9aed68ef61798fdfe43d41f695554c2a48084d403"}
Oct 30 22:31:31.554770 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.554745    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:31:31.663923 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.663885    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664019    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664070    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664123    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664163    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664216    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664248    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664328 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664299    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.664577 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.664327    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs\") pod \"2064d62c-1633-47c4-b455-354994115829\" (UID: \"2064d62c-1633-47c4-b455-354994115829\") "
Oct 30 22:31:31.665141 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.665115    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:31.665844 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.665627    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:31.666105 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.666087    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:31.667556 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.667531    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:31.668685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.668659    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:31.670568 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.670532    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:31:31.672769 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.672732    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "kube-api-access-fxzts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:31.672856 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.672838    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:31.673277 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.673243    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token" pod "2064d62c-1633-47c4-b455-354994115829" (UID: "2064d62c-1633-47c4-b455-354994115829"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767294    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxzts\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-kube-api-access-fxzts\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767337    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2064d62c-1633-47c4-b455-354994115829-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767354    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767370    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767385    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767418    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2064d62c-1633-47c4-b455-354994115829-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767431    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2064d62c-1633-47c4-b455-354994115829-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767445    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:31.768502 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:31.767460    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2064d62c-1633-47c4-b455-354994115829-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:32.367755 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:32.367727    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"
Oct 30 22:31:32.368218 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:32.367723    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5" event={"ID":"2064d62c-1633-47c4-b455-354994115829","Type":"ContainerDied","Data":"021e72b9c2a72980c74b8f363b994284660044a2f3ef8cbfb48e2def34fe6eb0"}
Oct 30 22:31:32.368218 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:32.367871    4536 scope.go:122] "RemoveContainer" containerID="8ce213841e3d182c4c21e0e9aed68ef61798fdfe43d41f695554c2a48084d403"
Oct 30 22:31:32.591587 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:32.591549    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"]
Oct 30 22:31:32.641459 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:32.641371    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c76d9e-openshift-default-5dddb76f45-mkjv5"]
Oct 30 22:31:33.165762 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:33.165731    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2064d62c-1633-47c4-b455-354994115829" path="/var/lib/kubelet/pods/2064d62c-1633-47c4-b455-354994115829/volumes"
Oct 30 22:31:39.009728 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:39.009690    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"]
Oct 30 22:31:39.010213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:39.009919    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" containerID="cri-o://f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db" gracePeriod=30
Oct 30 22:31:42.342655 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:42.342624    4536 patch_prober.go:29] interesting pod/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:31:42.343043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:42.342687    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:31:44.280437 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.280352    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:31:44.347646 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347611    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.347850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347668    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.347850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347714    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.347850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347737    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.347850 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347835    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.348039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347868    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.348039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347906    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.348039 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.347990    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.348163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.348053    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data\") pod \"e340d27b-4200-4797-b166-d003fc6ada89\" (UID: \"e340d27b-4200-4797-b166-d003fc6ada89\") "
Oct 30 22:31:44.348701 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.348665    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:44.348857 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.348745    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:44.349038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.348996    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:44.349107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.349033    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:44.349107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.349054    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:44.350295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.350269    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:31:44.350681 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.350656    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "kube-api-access-dtqv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:44.351070 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.351044    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:44.351194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.351174    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy" pod "e340d27b-4200-4797-b166-d003fc6ada89" (UID: "e340d27b-4200-4797-b166-d003fc6ada89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:44.449271 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449240    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtqv8\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-kube-api-access-dtqv8\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449271 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449269    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e340d27b-4200-4797-b166-d003fc6ada89-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449271 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449278    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e340d27b-4200-4797-b166-d003fc6ada89-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449287    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449297    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449305    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449314    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e340d27b-4200-4797-b166-d003fc6ada89-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449322    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.449515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.449330    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e340d27b-4200-4797-b166-d003fc6ada89-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:44.472745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.469887    4536 generic.go:356] "Generic (PLEG): container finished" podID="e340d27b-4200-4797-b166-d003fc6ada89" containerID="f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db" exitCode=0
Oct 30 22:31:44.472745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.469950    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"
Oct 30 22:31:44.472745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.469981    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" event={"ID":"e340d27b-4200-4797-b166-d003fc6ada89","Type":"ContainerDied","Data":"f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db"}
Oct 30 22:31:44.472745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.470048    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn" event={"ID":"e340d27b-4200-4797-b166-d003fc6ada89","Type":"ContainerDied","Data":"69c88028fb127bb8af826cbfa0057cd1484ff81dfe2730a163806e7cd03c3def"}
Oct 30 22:31:44.472745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.470080    4536 scope.go:122] "RemoveContainer" containerID="f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db"
Oct 30 22:31:44.486738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.486715    4536 scope.go:122] "RemoveContainer" containerID="f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db"
Oct 30 22:31:44.487130 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:44.487110    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db\": container with ID starting with f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db not found: ID does not exist" containerID="f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db"
Oct 30 22:31:44.487202 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.487139    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db"} err="failed to get container status \"f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db\": rpc error: code = NotFound desc = could not find container \"f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db\": container with ID starting with f46136cc1158172c8e9d3b69c341c3823acbcad890c3160c2ef80fdba60096db not found: ID does not exist"
Oct 30 22:31:44.637960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.637874    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"]
Oct 30 22:31:44.641111 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:44.641086    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-963c3a-openshift-default-8595756565-5xqgn"]
Oct 30 22:31:45.165710 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:31:45.165683    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e340d27b-4200-4797-b166-d003fc6ada89" path="/var/lib/kubelet/pods/e340d27b-4200-4797-b166-d003fc6ada89/volumes"
Oct 30 22:31:50.805854 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:31:50.805813    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:32:20.833493 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:32:20.833445    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:32:50.805828 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:32:50.805739    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:32:51.278312 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.278274    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"]
Oct 30 22:32:51.278517 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.278467    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" podUID="6380a3ed-6530-41d7-919d-e0e759f62513" containerName="maas-api" containerID="cri-o://04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c" gracePeriod=30
Oct 30 22:32:51.593203 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.593177    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:32:51.684955 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.684911    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs\") pod \"6380a3ed-6530-41d7-919d-e0e759f62513\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") "
Oct 30 22:32:51.685284 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.685258    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca\") pod \"6380a3ed-6530-41d7-919d-e0e759f62513\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") "
Oct 30 22:32:51.685459 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.685444    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv\" (UniqueName: \"kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv\") pod \"6380a3ed-6530-41d7-919d-e0e759f62513\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") "
Oct 30 22:32:51.685636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.685622    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\") pod \"6380a3ed-6530-41d7-919d-e0e759f62513\" (UID: \"6380a3ed-6530-41d7-919d-e0e759f62513\") "
Oct 30 22:32:51.688117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.687749    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca" pod "6380a3ed-6530-41d7-919d-e0e759f62513" (UID: "6380a3ed-6530-41d7-919d-e0e759f62513"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:51.694615 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.694585    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv" pod "6380a3ed-6530-41d7-919d-e0e759f62513" (UID: "6380a3ed-6530-41d7-919d-e0e759f62513"). InnerVolumeSpecName "kube-api-access-zjgzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:51.713646 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.710871    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls" pod "6380a3ed-6530-41d7-919d-e0e759f62513" (UID: "6380a3ed-6530-41d7-919d-e0e759f62513"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:51.716485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.716448    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs" pod "6380a3ed-6530-41d7-919d-e0e759f62513" (UID: "6380a3ed-6530-41d7-919d-e0e759f62513"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:51.788336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.788305    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:51.788336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.788337    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380a3ed-6530-41d7-919d-e0e759f62513-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:51.788336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.788347    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjgzv\" (UniqueName: \"kubernetes.io/projected/6380a3ed-6530-41d7-919d-e0e759f62513-kube-api-access-zjgzv\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:51.788592 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.788356    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380a3ed-6530-41d7-919d-e0e759f62513-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:51.861944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.861842    4536 generic.go:356] "Generic (PLEG): container finished" podID="6380a3ed-6530-41d7-919d-e0e759f62513" containerID="04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c" exitCode=0
Oct 30 22:32:51.861944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.861891    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"
Oct 30 22:32:51.862665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.861937    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" event={"ID":"6380a3ed-6530-41d7-919d-e0e759f62513","Type":"ContainerDied","Data":"04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c"}
Oct 30 22:32:51.862665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.862076    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w" event={"ID":"6380a3ed-6530-41d7-919d-e0e759f62513","Type":"ContainerDied","Data":"53c29917dd4382ed4bfd1a8bd47d9b6ce6c7fbac25749d820703327bac783167"}
Oct 30 22:32:51.862665 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.862110    4536 scope.go:122] "RemoveContainer" containerID="04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c"
Oct 30 22:32:51.872154 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.872138    4536 scope.go:122] "RemoveContainer" containerID="04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c"
Oct 30 22:32:51.872526 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:32:51.872488    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c\": container with ID starting with 04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c not found: ID does not exist" containerID="04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c"
Oct 30 22:32:51.872645 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:51.872533    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c"} err="failed to get container status \"04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c\": rpc error: code = NotFound desc = could not find container \"04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c\": container with ID starting with 04b6b46fbeb238830badfb816978db77e44700c4e550efb8b39f8a9066b7a74c not found: ID does not exist"
Oct 30 22:32:52.031029 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:52.030989    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"]
Oct 30 22:32:52.039350 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:52.039318    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-13b8d8-698bbf668f-br67w"]
Oct 30 22:32:53.165038 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:32:53.165008    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6380a3ed-6530-41d7-919d-e0e759f62513" path="/var/lib/kubelet/pods/6380a3ed-6530-41d7-919d-e0e759f62513/volumes"
Oct 30 22:33:13.438405 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.438351    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"]
Oct 30 22:33:13.439084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439062    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439088    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439101    4536 cpu_manager.go:414] "containerMap: removing container" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439109    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439130    4536 cpu_manager.go:414] "containerMap: removing container" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439138    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439149    4536 cpu_manager.go:414] "containerMap: removing container" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439156    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439169    4536 cpu_manager.go:414] "containerMap: removing container" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439176    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439192    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439199    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439209    4536 cpu_manager.go:414] "containerMap: removing container" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439215    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439230    4536 cpu_manager.go:414] "containerMap: removing container" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439237    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439254    4536 cpu_manager.go:414] "containerMap: removing container" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439260    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main"
Oct 30 22:33:13.439269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439271    4536 cpu_manager.go:414] "containerMap: removing container" podUID="6fed049f-6d70-47db-aeec-2bcbf2271ce9" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439278    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fed049f-6d70-47db-aeec-2bcbf2271ce9" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439308    4536 cpu_manager.go:414] "containerMap: removing container" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439315    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439324    4536 cpu_manager.go:414] "containerMap: removing container" podUID="70861c3f-0093-47d0-8f26-3f7967f40eb5" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439331    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70861c3f-0093-47d0-8f26-3f7967f40eb5" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439345    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439352    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439361    4536 cpu_manager.go:414] "containerMap: removing container" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439369    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439382    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439389    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439420    4536 cpu_manager.go:414] "containerMap: removing container" podUID="6380a3ed-6530-41d7-919d-e0e759f62513" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439427    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6380a3ed-6530-41d7-919d-e0e759f62513" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439441    4536 cpu_manager.go:414] "containerMap: removing container" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439448    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439553    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="081b695c-a7d3-4519-b0af-745bfcfa2ad1" containerName="main"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439567    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="f448fdd2-77cc-4c38-ba05-1078286a2a32" containerName="main"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439575    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fed049f-6d70-47db-aeec-2bcbf2271ce9" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439587    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439594    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439602    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="70861c3f-0093-47d0-8f26-3f7967f40eb5" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439611    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439621    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f1e0b03-63d1-4636-ab80-c8298ebe8a6f" containerName="main"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439628    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439636    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439645    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439654    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="6380a3ed-6530-41d7-919d-e0e759f62513" containerName="maas-api"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439664    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439672    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cb3e529-4595-4efd-8623-eb810c224b99" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439679    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439689    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="2064d62c-1633-47c4-b455-354994115829" containerName="istio-proxy"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439698    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439705    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e340d27b-4200-4797-b166-d003fc6ada89" containerName="istio-proxy"
Oct 30 22:33:13.439722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.439714    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.443084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.443069    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.487118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487080    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"]
Oct 30 22:33:13.487694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487664    4536 cpu_manager.go:414] "containerMap: removing container" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.487694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487695    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33db14df-ef7c-4008-a269-c0c36c8ea9af" containerName="cleanup"
Oct 30 22:33:13.487694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487705    4536 cpu_manager.go:414] "containerMap: removing container" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.487950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487710    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.487950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487719    4536 cpu_manager.go:414] "containerMap: removing container" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.487950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487723    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="843976e7-0a3e-4fe5-b192-8a16dcc316c1" containerName="cleanup"
Oct 30 22:33:13.487950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.487815    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa8dbbc5-31ec-480e-99f5-9aaba2eac22e" containerName="cleanup"
Oct 30 22:33:13.491196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.491176    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"]
Oct 30 22:33:13.491293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.491277    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.494817 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.494801    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.534091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.534064    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9qrfj\""
Oct 30 22:33:13.545229 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.545188    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"]
Oct 30 22:33:13.550688 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550663    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.550830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550699    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.550830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550731    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.550830 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550817    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.550939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550857    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.550939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550916    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551008 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550953    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551008 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.550998    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551107 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551035    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551169 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551108    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41528b-55be-45e7-9624-14885716b61c-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551169 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551143    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.551238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551190    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551238 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551218    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c24wl\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.551347 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551283    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.551347 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551334    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551435    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41528b-55be-45e7-9624-14885716b61c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551474    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41528b-55be-45e7-9624-14885716b61c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551542 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551528    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b9h9\" (UniqueName: \"kubernetes.io/projected/cf41528b-55be-45e7-9624-14885716b61c-kube-api-access-4b9h9\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551552    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551629 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551604    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551662    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551703    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vwjq\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551779 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551749    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551779 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551770    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.551884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551798    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.551884 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551854    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.551980 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.551894    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.603538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.603504    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"]
Oct 30 22:33:13.645781 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.645745    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"]
Oct 30 22:33:13.653454 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653422    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653584 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653468    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.653584 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653507    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41528b-55be-45e7-9624-14885716b61c-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653584 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653537    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.653584 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653565    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.653742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653600    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c24wl\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.653742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653661    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.653742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653697    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653743    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41528b-55be-45e7-9624-14885716b61c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653775    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41528b-55be-45e7-9624-14885716b61c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653812    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b9h9\" (UniqueName: \"kubernetes.io/projected/cf41528b-55be-45e7-9624-14885716b61c-kube-api-access-4b9h9\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653836    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.653933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653864    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.653933 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653897    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.654035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653936    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vwjq\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.654035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.653969    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.654035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654000    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.654035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654029    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.654168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654055    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.654168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654082    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.654168 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654162    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.654267 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654186    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.654267 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654218    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.654267 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654245    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.654357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654284    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.654357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654309    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.654357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654340    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.654857 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.654835    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.655177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.655157    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.655452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.655433    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.656064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.656036    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.656351 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.656323    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.656499 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.656349    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.656499 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.656390    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.656736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.656710    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.656996 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.656972    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.657077 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.657060    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.657157 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.657134    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.657255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.657232    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.658066 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.658042    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.659255 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.658778    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.659380 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.659180    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf41528b-55be-45e7-9624-14885716b61c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.659380 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.659175    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.659826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.659804    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf41528b-55be-45e7-9624-14885716b61c-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.660175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.660155    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.660651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.660632    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.660878 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.660860    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.661778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.661763    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf41528b-55be-45e7-9624-14885716b61c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.718541 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.718510    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf41528b-55be-45e7-9624-14885716b61c-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.718968 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.718909    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.719431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.719384    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vwjq\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.719833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.719805    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c24wl\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl\") pod \"maas-default-gateway-openshift-default-76597d6855-4fbhf\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.720051 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.720028    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-4gjwd\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.721342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.721325    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b9h9\" (UniqueName: \"kubernetes.io/projected/cf41528b-55be-45e7-9624-14885716b61c-kube-api-access-4b9h9\") pod \"maas-default-gateway-openshift-default-76597d6855-tqtgt\" (UID: \"cf41528b-55be-45e7-9624-14885716b61c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:13.765217 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.765192    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:13.805761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.805727    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:13.813137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:13.813109    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:14.202472 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.202438    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"]
Oct 30 22:33:14.246211 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:33:14.246180    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0b04b6_d98a_4838_8641_2ed9778d0865.slice/crio-0101318c9092ba766fd4a200fef6f328c69323d3d436cf5ce5a995936766edf2 WatchSource:0}: Error finding container 0101318c9092ba766fd4a200fef6f328c69323d3d436cf5ce5a995936766edf2: Status 404 returned error can't find the container with id 0101318c9092ba766fd4a200fef6f328c69323d3d436cf5ce5a995936766edf2
Oct 30 22:33:14.248683 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.248650    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.248769 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.248721    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.248769 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.248749    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.264759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.264738    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"]
Oct 30 22:33:14.265388 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:33:14.265345    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf41528b_55be_45e7_9624_14885716b61c.slice/crio-5fffe930747ff7abb4279acf0b72b8cb0018a5d3e7298ef285a8fa121ce60378 WatchSource:0}: Error finding container 5fffe930747ff7abb4279acf0b72b8cb0018a5d3e7298ef285a8fa121ce60378: Status 404 returned error can't find the container with id 5fffe930747ff7abb4279acf0b72b8cb0018a5d3e7298ef285a8fa121ce60378
Oct 30 22:33:14.268282 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.268230    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.268422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.268325    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.268422 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.268371    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.311794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.311770    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"]
Oct 30 22:33:14.314708 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:33:14.314680    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b4f057_54f5_4aca_a01d_d07c1dcc7500.slice/crio-ba12b3e3a65fab0473608ce647813da84d57061eb81b493c350c3174e742c2be WatchSource:0}: Error finding container ba12b3e3a65fab0473608ce647813da84d57061eb81b493c350c3174e742c2be: Status 404 returned error can't find the container with id ba12b3e3a65fab0473608ce647813da84d57061eb81b493c350c3174e742c2be
Oct 30 22:33:14.317768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.317588    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.317768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.317674    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:14.317768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:14.317715    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:33:15.028431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.021945    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" event={"ID":"cf41528b-55be-45e7-9624-14885716b61c","Type":"ContainerStarted","Data":"0a8963e33bbd44170c0e2e28456e368d8d77ee05a6c8b9cf3fcc34a734ef0795"}
Oct 30 22:33:15.028431 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.022013    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" event={"ID":"cf41528b-55be-45e7-9624-14885716b61c","Type":"ContainerStarted","Data":"5fffe930747ff7abb4279acf0b72b8cb0018a5d3e7298ef285a8fa121ce60378"}
Oct 30 22:33:15.032461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.032003    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" event={"ID":"35b4f057-54f5-4aca-a01d-d07c1dcc7500","Type":"ContainerStarted","Data":"5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5"}
Oct 30 22:33:15.032461 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.032072    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" event={"ID":"35b4f057-54f5-4aca-a01d-d07c1dcc7500","Type":"ContainerStarted","Data":"ba12b3e3a65fab0473608ce647813da84d57061eb81b493c350c3174e742c2be"}
Oct 30 22:33:15.043814 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.043776    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" event={"ID":"df0b04b6-d98a-4838-8641-2ed9778d0865","Type":"ContainerStarted","Data":"bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145"}
Oct 30 22:33:15.044062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.044045    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" event={"ID":"df0b04b6-d98a-4838-8641-2ed9778d0865","Type":"ContainerStarted","Data":"0101318c9092ba766fd4a200fef6f328c69323d3d436cf5ce5a995936766edf2"}
Oct 30 22:33:15.164215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.164154    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podStartSLOduration=2.164135269 podStartE2EDuration="2.164135269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:15.162786503 +0000 UTC m=+2326.150594961" watchObservedRunningTime="2026-10-30 22:33:15.164135269 +0000 UTC m=+2326.151943537"
Oct 30 22:33:15.300722 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.300596    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podStartSLOduration=2.300575051 podStartE2EDuration="2.300575051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:15.297722115 +0000 UTC m=+2326.285530379" watchObservedRunningTime="2026-10-30 22:33:15.300575051 +0000 UTC m=+2326.288383316"
Oct 30 22:33:15.433130 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.433065    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podStartSLOduration=2.433044515 podStartE2EDuration="2.433044515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:15.429086081 +0000 UTC m=+2326.416894345" watchObservedRunningTime="2026-10-30 22:33:15.433044515 +0000 UTC m=+2326.420852779"
Oct 30 22:33:15.766494 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.766464    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:15.767093 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.767071    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:15.767201 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.767123    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:15.807636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.807609    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:15.807989 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.807965    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:15.808163 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.808139    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:15.828423 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.819955    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:15.829218 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.829186    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:15.829605 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:15.829574    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:16.766261 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:16.766223    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:16.766697 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:16.766294    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:16.815433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:16.814796    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:16.815433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:16.814856    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:16.815433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:16.815204    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:16.815433 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:16.815239    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:17.766118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:17.766082    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:17.766294 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:17.766132    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:17.809424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:17.808854    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:17.809424 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:17.808913    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:17.817429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:17.814509    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:17.817429 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:17.814569    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:18.765662 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:18.765626    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:18.765853 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:18.765685    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:18.807421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:18.806832    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:18.807421 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:18.806899    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:18.820430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:18.815808    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:18.820430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:18.815875    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:19.766496 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:19.766462    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:19.766763 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:19.766739    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:19.809912 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:19.809876    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:19.810609 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:19.810561    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:19.814881 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:19.814854    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:19.815083 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:19.815059    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:20.766300 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:20.766260    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:20.766509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:20.766322    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:20.806545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:20.806511    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:20.806712 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:20.806571    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:20.807782 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:33:20.807745    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:33:20.813848 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:20.813814    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:20.814196 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:20.813886    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:21.765902 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:21.765857    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4gjwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Oct 30 22:33:21.766138 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:21.765930    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Oct 30 22:33:21.806668 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:21.806629    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-4fbhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:33:21.806852 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:21.806696    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:33:21.814062 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:21.814031    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 22:33:21.814383 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:21.814091    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 22:33:23.563652 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.563619    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:23.564261 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.564005    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:23.565512 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.565487    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:33:23.608444 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.608382    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:23.608932 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.608906    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:23.609795 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.609778    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:33:23.662872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.662840    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:23.663332 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.663309    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:23.664279 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:23.664259    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt"
Oct 30 22:33:30.030482 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:30.030444    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"]
Oct 30 22:33:30.031008 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:30.030658    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy" containerID="cri-o://c96b9d4cd0ccee819f04fbd3ce55463e55ffb9f3f9658d11e13d34ff51c93ef4" gracePeriod=30
Oct 30 22:33:30.055257 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:30.055226    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"]
Oct 30 22:33:30.055511 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:30.055479    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy" containerID="cri-o://8201347c6a9a5a5f5a0f415c13bc311eb586b80ff87163fdde0b93cb74583934" gracePeriod=30
Oct 30 22:33:35.206159 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.206120    4536 generic.go:356] "Generic (PLEG): container finished" podID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerID="c96b9d4cd0ccee819f04fbd3ce55463e55ffb9f3f9658d11e13d34ff51c93ef4" exitCode=0
Oct 30 22:33:35.206619 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.206278    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" event={"ID":"f7c9c649-98e4-41fc-a3bc-9adab9962a3f","Type":"ContainerDied","Data":"c96b9d4cd0ccee819f04fbd3ce55463e55ffb9f3f9658d11e13d34ff51c93ef4"}
Oct 30 22:33:35.208089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.207974    4536 generic.go:356] "Generic (PLEG): container finished" podID="a86de333-9735-452c-aa27-c15903aa2717" containerID="8201347c6a9a5a5f5a0f415c13bc311eb586b80ff87163fdde0b93cb74583934" exitCode=0
Oct 30 22:33:35.208089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.208039    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" event={"ID":"a86de333-9735-452c-aa27-c15903aa2717","Type":"ContainerDied","Data":"8201347c6a9a5a5f5a0f415c13bc311eb586b80ff87163fdde0b93cb74583934"}
Oct 30 22:33:35.331016 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.330993    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:33:35.358715 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.358685    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:33:35.472280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472251    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.472280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472286    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.472552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472335    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.472552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472372    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.472552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472446    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.472552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472481    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.472552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472515    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.472768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472553    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.472768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472583    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.472768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472650    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.472768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472703    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.472768 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472738    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.473034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472761    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.473034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472783    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:33:35.473034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472787    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.473034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472844    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.473034 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.472942    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.473295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473105    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:33:35.473295 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473257    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket\") pod \"a86de333-9735-452c-aa27-c15903aa2717\" (UID: \"a86de333-9735-452c-aa27-c15903aa2717\") "
Oct 30 22:33:35.473434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473300    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.473434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473332    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs\") pod \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\" (UID: \"f7c9c649-98e4-41fc-a3bc-9adab9962a3f\") "
Oct 30 22:33:35.473652 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473623    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473848    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473885    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.473967    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474051    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474070    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474078    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474083    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.474118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474088    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.474651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474179    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.474651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474227    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a86de333-9735-452c-aa27-c15903aa2717-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.474651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474278    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.474651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.474386    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.476412 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.476290    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:35.476412 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.476355    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "kube-api-access-4wscn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:35.476601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.476541    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.476601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.476590    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:35.476765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.476738    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:35.476979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.476953    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "kube-api-access-t4phl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:35.477074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.477004    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo" pod "f7c9c649-98e4-41fc-a3bc-9adab9962a3f" (UID: "f7c9c649-98e4-41fc-a3bc-9adab9962a3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:33:35.477886 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.477865    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo" pod "a86de333-9735-452c-aa27-c15903aa2717" (UID: "a86de333-9735-452c-aa27-c15903aa2717"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575709    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575734    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575746    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a86de333-9735-452c-aa27-c15903aa2717-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575756    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4phl\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-kube-api-access-t4phl\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575765    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575776    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575784    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575792    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.575796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575801    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7c9c649-98e4-41fc-a3bc-9adab9962a3f-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.576372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575810    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.576372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575819    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wscn\" (UniqueName: \"kubernetes.io/projected/a86de333-9735-452c-aa27-c15903aa2717-kube-api-access-4wscn\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:35.576372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:35.575827    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a86de333-9735-452c-aa27-c15903aa2717-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:36.214773 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.214743    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"
Oct 30 22:33:36.215259 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.214771    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq" event={"ID":"a86de333-9735-452c-aa27-c15903aa2717","Type":"ContainerDied","Data":"35fbb0c90c5539235dafb4913ea7314d6dab88e84122fffbeba3a460866aa8ae"}
Oct 30 22:33:36.215259 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.214834    4536 scope.go:122] "RemoveContainer" containerID="8201347c6a9a5a5f5a0f415c13bc311eb586b80ff87163fdde0b93cb74583934"
Oct 30 22:33:36.216622 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.216469    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv" event={"ID":"f7c9c649-98e4-41fc-a3bc-9adab9962a3f","Type":"ContainerDied","Data":"fdd6861c9412c52f099154d2c7442bce3906bd5490089ff4ab0ba78bb9f96530"}
Oct 30 22:33:36.216622 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.216585    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"
Oct 30 22:33:36.226752 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.226728    4536 scope.go:122] "RemoveContainer" containerID="c96b9d4cd0ccee819f04fbd3ce55463e55ffb9f3f9658d11e13d34ff51c93ef4"
Oct 30 22:33:36.383316 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.383242    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"]
Oct 30 22:33:36.433137 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.433105    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-hgdsv"]
Oct 30 22:33:36.502418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.502370    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"]
Oct 30 22:33:36.512957 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:36.512931    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-77cb1f-openshift-default-57f8c95f4c-8w9nq"]
Oct 30 22:33:37.165194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:37.165163    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a86de333-9735-452c-aa27-c15903aa2717" path="/var/lib/kubelet/pods/a86de333-9735-452c-aa27-c15903aa2717/volumes"
Oct 30 22:33:37.165650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:33:37.165636    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" path="/var/lib/kubelet/pods/f7c9c649-98e4-41fc-a3bc-9adab9962a3f/volumes"
Oct 30 22:33:50.805114 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:33:50.805075    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:34:04.812198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:04.812158    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"]
Oct 30 22:34:13.411970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:13.411887    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"]
Oct 30 22:34:13.412636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:13.412593    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" containerID="cri-o://3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3" gracePeriod=30
Oct 30 22:34:14.150286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:14.150237    4536 patch_prober.go:29] interesting pod/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:34:14.150506 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:14.150340    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:34:18.671883 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.671861    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:34:18.827170 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827136    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827201    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827286    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827336    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827441    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827470    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827465    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:18.827540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827501    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827525    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:18.827851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827604    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827632    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx\") pod \"99a943de-9159-4e5b-85cb-f2ff60788d05\" (UID: \"99a943de-9159-4e5b-85cb-f2ff60788d05\") "
Oct 30 22:34:18.827851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827636    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:18.827851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.827754    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:18.828077 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.828045    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:18.828307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.828074    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.828307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.828096    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.828307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.828108    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.828307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.828120    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.829773 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.829750    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:34:18.829891 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.829792    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:18.830215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.830186    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:18.830263 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.830217    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx" pod "99a943de-9159-4e5b-85cb-f2ff60788d05" (UID: "99a943de-9159-4e5b-85cb-f2ff60788d05"). InnerVolumeSpecName "kube-api-access-bjgjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:18.929409 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.929373    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.929590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.929428    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.929590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.929437    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99a943de-9159-4e5b-85cb-f2ff60788d05-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.929590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.929446    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjgjx\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-kube-api-access-bjgjx\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:18.929590 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:18.929457    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99a943de-9159-4e5b-85cb-f2ff60788d05-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:19.461692 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.461658    4536 generic.go:356] "Generic (PLEG): container finished" podID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerID="3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3" exitCode=0
Oct 30 22:34:19.461924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.461717    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" event={"ID":"99a943de-9159-4e5b-85cb-f2ff60788d05","Type":"ContainerDied","Data":"3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3"}
Oct 30 22:34:19.462056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.461965    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw" event={"ID":"99a943de-9159-4e5b-85cb-f2ff60788d05","Type":"ContainerDied","Data":"0b579b687e5348a8c758c354d1c593a5de857d4f375bcc86bb26f86668e6ead1"}
Oct 30 22:34:19.462056 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.462000    4536 scope.go:122] "RemoveContainer" containerID="3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3"
Oct 30 22:34:19.462148 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.461753    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"
Oct 30 22:34:19.475681 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.475662    4536 scope.go:122] "RemoveContainer" containerID="3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3"
Oct 30 22:34:19.479876 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:34:19.479848    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3\": container with ID starting with 3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3 not found: ID does not exist" containerID="3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3"
Oct 30 22:34:19.480004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.479887    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3"} err="failed to get container status \"3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3\": rpc error: code = NotFound desc = could not find container \"3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3\": container with ID starting with 3b0dd9bd5cb9018aaf3d97697d2576c65a6d3d86c24ad252ee2cbe6a880ffda3 not found: ID does not exist"
Oct 30 22:34:19.684099 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.684059    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"]
Oct 30 22:34:19.687774 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.687748    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-fjgnw"]
Oct 30 22:34:19.833240 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:19.833154    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld" podUID="fedefc96-6844-46b7-b33f-b06e465914fb" containerName="main" containerID="cri-o://129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d" gracePeriod=45
Oct 30 22:34:20.110565 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.110533    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:34:20.246785 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.246751    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs\") pod \"fedefc96-6844-46b7-b33f-b06e465914fb\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") "
Oct 30 22:34:20.246949 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.246806    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache\") pod \"fedefc96-6844-46b7-b33f-b06e465914fb\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") "
Oct 30 22:34:20.247012 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.246956    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f\" (UniqueName: \"kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f\") pod \"fedefc96-6844-46b7-b33f-b06e465914fb\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") "
Oct 30 22:34:20.247089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247072    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm\") pod \"fedefc96-6844-46b7-b33f-b06e465914fb\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") "
Oct 30 22:34:20.247155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247119    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir\") pod \"fedefc96-6844-46b7-b33f-b06e465914fb\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") "
Oct 30 22:34:20.247155 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247132    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache" pod "fedefc96-6844-46b7-b33f-b06e465914fb" (UID: "fedefc96-6844-46b7-b33f-b06e465914fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.247269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247161    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home\") pod \"fedefc96-6844-46b7-b33f-b06e465914fb\" (UID: \"fedefc96-6844-46b7-b33f-b06e465914fb\") "
Oct 30 22:34:20.247570 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247542    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir" pod "fedefc96-6844-46b7-b33f-b06e465914fb" (UID: "fedefc96-6844-46b7-b33f-b06e465914fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.247703 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247645    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home" pod "fedefc96-6844-46b7-b33f-b06e465914fb" (UID: "fedefc96-6844-46b7-b33f-b06e465914fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.247960 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247938    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.248050 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247965    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.248050 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.247981    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.249786 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.249760    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs" pod "fedefc96-6844-46b7-b33f-b06e465914fb" (UID: "fedefc96-6844-46b7-b33f-b06e465914fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:20.249992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.249971    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f" pod "fedefc96-6844-46b7-b33f-b06e465914fb" (UID: "fedefc96-6844-46b7-b33f-b06e465914fb"). InnerVolumeSpecName "kube-api-access-ms69f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:20.250090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.250003    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm" pod "fedefc96-6844-46b7-b33f-b06e465914fb" (UID: "fedefc96-6844-46b7-b33f-b06e465914fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:20.350344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.350212    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedefc96-6844-46b7-b33f-b06e465914fb-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.350344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.350247    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms69f\" (UniqueName: \"kubernetes.io/projected/fedefc96-6844-46b7-b33f-b06e465914fb-kube-api-access-ms69f\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.350344 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.350263    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedefc96-6844-46b7-b33f-b06e465914fb-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:20.468025 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.467991    4536 generic.go:356] "Generic (PLEG): container finished" podID="fedefc96-6844-46b7-b33f-b06e465914fb" containerID="129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d" exitCode=0
Oct 30 22:34:20.468175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.468036    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld" event={"ID":"fedefc96-6844-46b7-b33f-b06e465914fb","Type":"ContainerDied","Data":"129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d"}
Oct 30 22:34:20.468175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.468050    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"
Oct 30 22:34:20.468175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.468081    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld" event={"ID":"fedefc96-6844-46b7-b33f-b06e465914fb","Type":"ContainerDied","Data":"59b11047b953f2451baa533c54c65d8fc0c3c89b18ac39411d04f530ef66182f"}
Oct 30 22:34:20.468175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.468099    4536 scope.go:122] "RemoveContainer" containerID="129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d"
Oct 30 22:34:20.478725 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.478708    4536 scope.go:122] "RemoveContainer" containerID="129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d"
Oct 30 22:34:20.478997 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:34:20.478980    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d\": container with ID starting with 129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d not found: ID does not exist" containerID="129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d"
Oct 30 22:34:20.479063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.479007    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d"} err="failed to get container status \"129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d\": rpc error: code = NotFound desc = could not find container \"129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d\": container with ID starting with 129d4be4f452c57c0dd5f988b48a7ccba902fbc4835b1226529cfc9c41c72a0d not found: ID does not exist"
Oct 30 22:34:20.639528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.639442    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"]
Oct 30 22:34:20.645154 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:20.645128    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-18515933/e2e-lifecycle-model-18515933-kserve-55c864d877-5h4ld"]
Oct 30 22:34:20.805546 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:34:20.805514    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:34:21.164161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:21.164132    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" path="/var/lib/kubelet/pods/99a943de-9159-4e5b-85cb-f2ff60788d05/volumes"
Oct 30 22:34:21.164582 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:21.164568    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fedefc96-6844-46b7-b33f-b06e465914fb" path="/var/lib/kubelet/pods/fedefc96-6844-46b7-b33f-b06e465914fb/volumes"
Oct 30 22:34:45.130526 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:34:45.130491    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"]
Oct 30 22:34:50.837141 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:34:50.837077    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:35:00.151144 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.151101    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" containerID="cri-o://8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92" gracePeriod=45
Oct 30 22:35:00.212188 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.212153    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": dial tcp 10.133.0.50:8000: connect: connection refused"
Oct 30 22:35:00.427410 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.427371    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:35:00.516784 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.516755    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache\") pod \"ddd044c2-e404-4b13-9c32-067038dc37a6\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") "
Oct 30 22:35:00.516965 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.516846    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm\") pod \"ddd044c2-e404-4b13-9c32-067038dc37a6\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") "
Oct 30 22:35:00.516965 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.516923    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs\") pod \"ddd044c2-e404-4b13-9c32-067038dc37a6\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") "
Oct 30 22:35:00.516965 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.516950    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home\") pod \"ddd044c2-e404-4b13-9c32-067038dc37a6\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") "
Oct 30 22:35:00.517096 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517026    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache" pod "ddd044c2-e404-4b13-9c32-067038dc37a6" (UID: "ddd044c2-e404-4b13-9c32-067038dc37a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:00.517177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517159    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw\" (UniqueName: \"kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw\") pod \"ddd044c2-e404-4b13-9c32-067038dc37a6\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") "
Oct 30 22:35:00.517234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517198    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir\") pod \"ddd044c2-e404-4b13-9c32-067038dc37a6\" (UID: \"ddd044c2-e404-4b13-9c32-067038dc37a6\") "
Oct 30 22:35:00.517234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517220    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home" pod "ddd044c2-e404-4b13-9c32-067038dc37a6" (UID: "ddd044c2-e404-4b13-9c32-067038dc37a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:00.517524 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517500    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir" pod "ddd044c2-e404-4b13-9c32-067038dc37a6" (UID: "ddd044c2-e404-4b13-9c32-067038dc37a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:00.517702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517687    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:00.517745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517705    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:00.517745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.517713    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:00.519311 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.519278    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm" pod "ddd044c2-e404-4b13-9c32-067038dc37a6" (UID: "ddd044c2-e404-4b13-9c32-067038dc37a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:00.519438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.519323    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs" pod "ddd044c2-e404-4b13-9c32-067038dc37a6" (UID: "ddd044c2-e404-4b13-9c32-067038dc37a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:00.519438 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.519375    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw" pod "ddd044c2-e404-4b13-9c32-067038dc37a6" (UID: "ddd044c2-e404-4b13-9c32-067038dc37a6"). InnerVolumeSpecName "kube-api-access-lgwgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:00.535833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.535809    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"]
Oct 30 22:35:00.536351 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536336    4536 cpu_manager.go:414] "containerMap: removing container" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy"
Oct 30 22:35:00.536351 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536351    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536365    4536 cpu_manager.go:414] "containerMap: removing container" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536370    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536381    4536 cpu_manager.go:414] "containerMap: removing container" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536386    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536417    4536 cpu_manager.go:414] "containerMap: removing container" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536423    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536439    4536 cpu_manager.go:414] "containerMap: removing container" podUID="fedefc96-6844-46b7-b33f-b06e465914fb" containerName="main"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536443    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fedefc96-6844-46b7-b33f-b06e465914fb" containerName="main"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536507    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="99a943de-9159-4e5b-85cb-f2ff60788d05" containerName="istio-proxy"
Oct 30 22:35:00.536516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536518    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerName="main"
Oct 30 22:35:00.536864 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536526    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="fedefc96-6844-46b7-b33f-b06e465914fb" containerName="main"
Oct 30 22:35:00.536864 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536533    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="a86de333-9735-452c-aa27-c15903aa2717" containerName="istio-proxy"
Oct 30 22:35:00.536864 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.536546    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7c9c649-98e4-41fc-a3bc-9adab9962a3f" containerName="istio-proxy"
Oct 30 22:35:00.539989 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.539971    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.594091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.594063    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e54161-openshift-default-dockercfg-dc5xx\""
Oct 30 22:35:00.618946 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.618913    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.618973    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619031    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619182 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619155    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619375 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619218    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619375 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619254    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619375 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619284    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtwxv\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619375 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619320    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619375 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619358    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.619663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619561    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddd044c2-e404-4b13-9c32-067038dc37a6-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:00.619663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619588    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgwgw\" (UniqueName: \"kubernetes.io/projected/ddd044c2-e404-4b13-9c32-067038dc37a6-kube-api-access-lgwgw\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:00.619663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.619604    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddd044c2-e404-4b13-9c32-067038dc37a6-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:00.635012 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.634988    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"]
Oct 30 22:35:00.699974 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.699938    4536 generic.go:356] "Generic (PLEG): container finished" podID="ddd044c2-e404-4b13-9c32-067038dc37a6" containerID="8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92" exitCode=0
Oct 30 22:35:00.700372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.699999    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"
Oct 30 22:35:00.700372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.700142    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" event={"ID":"ddd044c2-e404-4b13-9c32-067038dc37a6","Type":"ContainerDied","Data":"8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92"}
Oct 30 22:35:00.700372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.700179    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9" event={"ID":"ddd044c2-e404-4b13-9c32-067038dc37a6","Type":"ContainerDied","Data":"1049c1939fedbca82f7744b36365b2f97903c011f88987ed66f5d8d03b5e1011"}
Oct 30 22:35:00.700372 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.700204    4536 scope.go:122] "RemoveContainer" containerID="8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92"
Oct 30 22:35:00.712757 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.712735    4536 scope.go:122] "RemoveContainer" containerID="8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92"
Oct 30 22:35:00.713085 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:35:00.713065    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92\": container with ID starting with 8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92 not found: ID does not exist" containerID="8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92"
Oct 30 22:35:00.713190 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.713091    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92"} err="failed to get container status \"8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92\": rpc error: code = NotFound desc = could not find container \"8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92\": container with ID starting with 8197692a6c20ce582709e114421ec2d4717b31e41a3ea83cc79827e3d850fc92 not found: ID does not exist"
Oct 30 22:35:00.720831 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.720797    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.720955 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.720939    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.720981    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721010    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721140 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721094    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721200 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721161    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721200 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721194    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721221    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtwxv\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721220    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721646 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721255    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721332    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721342    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721741 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721477    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.721975 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.721951    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.723791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.723773    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.724020 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.723998    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.784102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.784070    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.784263 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.784190    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtwxv\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv\") pod \"e2e-inf-b-e54161-openshift-default-7899496757-gwjzf\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") " pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.852749 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.852710    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:00.873680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.873647    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"]
Oct 30 22:35:00.881519 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:00.881491    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-075cbb/distinct-w5-075cbb-kserve-69ff9b8d5-drlw9"]
Oct 30 22:35:01.183561 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.181792    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddd044c2-e404-4b13-9c32-067038dc37a6" path="/var/lib/kubelet/pods/ddd044c2-e404-4b13-9c32-067038dc37a6/volumes"
Oct 30 22:35:01.291911 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.291885    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"]
Oct 30 22:35:01.293849 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:35:01.293824    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740b2afb_173f_4c0e_a22e_892cb69da226.slice/crio-8d60479b0e60864afff8ebef5f586bbb202510763a6aa81263372861ace9b49a WatchSource:0}: Error finding container 8d60479b0e60864afff8ebef5f586bbb202510763a6aa81263372861ace9b49a: Status 404 returned error can't find the container with id 8d60479b0e60864afff8ebef5f586bbb202510763a6aa81263372861ace9b49a
Oct 30 22:35:01.296288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.296247    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:35:01.296419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.296324    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:35:01.296419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.296352    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:35:01.708245 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.708215    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" event={"ID":"740b2afb-173f-4c0e-a22e-892cb69da226","Type":"ContainerStarted","Data":"42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74"}
Oct 30 22:35:01.708559 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.708280    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" event={"ID":"740b2afb-173f-4c0e-a22e-892cb69da226","Type":"ContainerStarted","Data":"8d60479b0e60864afff8ebef5f586bbb202510763a6aa81263372861ace9b49a"}
Oct 30 22:35:01.837121 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.837066    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podStartSLOduration=1.837051486 podStartE2EDuration="1.837051486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:01.835896367 +0000 UTC m=+2432.823704654" watchObservedRunningTime="2026-10-30 22:35:01.837051486 +0000 UTC m=+2432.824859751"
Oct 30 22:35:01.853103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:01.853066    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:02.855380 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:02.855345    4536 patch_prober.go:29] interesting pod/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:35:02.855824 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:02.855432    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:35:03.853670 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:03.853632    4536 patch_prober.go:29] interesting pod/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:35:03.853873 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:03.853692    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:35:04.853288 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:04.853252    4536 patch_prober.go:29] interesting pod/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:35:04.853705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:04.853312    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:35:05.853089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:05.853054    4536 patch_prober.go:29] interesting pod/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:35:05.853269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:05.853118    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:35:06.853677 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:06.853644    4536 patch_prober.go:29] interesting pod/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 22:35:06.854074 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:06.853705    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 22:35:07.858386 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:07.858357    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:07.858796 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:07.858743    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:07.859500 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:07.859479    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:35:20.810386 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:35:20.810349    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:35:41.376024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.375936    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"]
Oct 30 22:35:41.376489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.376113    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" podUID="a8737966-b199-4970-baa9-922ee6f0b015" containerName="maas-api" containerID="cri-o://9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c" gracePeriod=30
Oct 30 22:35:41.674149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.674121    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:35:41.761934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.761894    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca\") pod \"a8737966-b199-4970-baa9-922ee6f0b015\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") "
Oct 30 22:35:41.762145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.761982    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs\") pod \"a8737966-b199-4970-baa9-922ee6f0b015\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") "
Oct 30 22:35:41.762145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.762026    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599\" (UniqueName: \"kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599\") pod \"a8737966-b199-4970-baa9-922ee6f0b015\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") "
Oct 30 22:35:41.762145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.762071    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\") pod \"a8737966-b199-4970-baa9-922ee6f0b015\" (UID: \"a8737966-b199-4970-baa9-922ee6f0b015\") "
Oct 30 22:35:41.797419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.787544    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca" pod "a8737966-b199-4970-baa9-922ee6f0b015" (UID: "a8737966-b199-4970-baa9-922ee6f0b015"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:41.797419 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.789050    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls" pod "a8737966-b199-4970-baa9-922ee6f0b015" (UID: "a8737966-b199-4970-baa9-922ee6f0b015"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:41.846205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.846130    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599" pod "a8737966-b199-4970-baa9-922ee6f0b015" (UID: "a8737966-b199-4970-baa9-922ee6f0b015"). InnerVolumeSpecName "kube-api-access-96599". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:41.846714 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.846689    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs" pod "a8737966-b199-4970-baa9-922ee6f0b015" (UID: "a8737966-b199-4970-baa9-922ee6f0b015"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:41.864186 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.864163    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:41.864293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.864208    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96599\" (UniqueName: \"kubernetes.io/projected/a8737966-b199-4970-baa9-922ee6f0b015-kube-api-access-96599\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:41.864293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.864219    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8737966-b199-4970-baa9-922ee6f0b015-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:41.864293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.864228    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8737966-b199-4970-baa9-922ee6f0b015-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:41.993302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.993270    4536 generic.go:356] "Generic (PLEG): container finished" podID="a8737966-b199-4970-baa9-922ee6f0b015" containerID="9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c" exitCode=0
Oct 30 22:35:41.993533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.993324    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"
Oct 30 22:35:41.993533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.993327    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" event={"ID":"a8737966-b199-4970-baa9-922ee6f0b015","Type":"ContainerDied","Data":"9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c"}
Oct 30 22:35:41.993533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.993496    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm" event={"ID":"a8737966-b199-4970-baa9-922ee6f0b015","Type":"ContainerDied","Data":"7939a5554d2496c3290042b7a4d797f139f94bb7ebadcbb60c582f8c30a07d39"}
Oct 30 22:35:41.993533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:41.993527    4536 scope.go:122] "RemoveContainer" containerID="9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c"
Oct 30 22:35:42.006944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:42.006927    4536 scope.go:122] "RemoveContainer" containerID="9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c"
Oct 30 22:35:42.007220 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:35:42.007199    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c\": container with ID starting with 9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c not found: ID does not exist" containerID="9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c"
Oct 30 22:35:42.007290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:42.007229    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c"} err="failed to get container status \"9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c\": rpc error: code = NotFound desc = could not find container \"9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c\": container with ID starting with 9d09f793a623529009f3aa6465ae08cb080aca1c96ea2ea3804a1301a580460c not found: ID does not exist"
Oct 30 22:35:42.163919 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:42.163889    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"]
Oct 30 22:35:42.171456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:42.171432    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-075cbb-7487dd9d65-hmqfm"]
Oct 30 22:35:43.164769 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:43.164738    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8737966-b199-4970-baa9-922ee6f0b015" path="/var/lib/kubelet/pods/a8737966-b199-4970-baa9-922ee6f0b015/volumes"
Oct 30 22:35:50.805310 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:35:50.805271    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:35:55.885199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:35:55.885159    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"]
Oct 30 22:36:10.906296 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:10.906234    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main" containerID="cri-o://7f8834e931b5687c10122fa3b1b54117ff07eaf8ecf94d5bd32ba6e8ab461467" gracePeriod=45
Oct 30 22:36:11.153727 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.153697    4536 generic.go:356] "Generic (PLEG): container finished" podID="5799f173-0882-4895-8049-087dfac11054" containerID="7f8834e931b5687c10122fa3b1b54117ff07eaf8ecf94d5bd32ba6e8ab461467" exitCode=0
Oct 30 22:36:11.153921 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.153778    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" event={"ID":"5799f173-0882-4895-8049-087dfac11054","Type":"ContainerDied","Data":"7f8834e931b5687c10122fa3b1b54117ff07eaf8ecf94d5bd32ba6e8ab461467"}
Oct 30 22:36:11.153921 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.153825    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg" event={"ID":"5799f173-0882-4895-8049-087dfac11054","Type":"ContainerDied","Data":"cbd18f243cb4fe7944e05d8a85bf79dbc0955719504f5b476e1b0af83bad6c65"}
Oct 30 22:36:11.153921 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.153840    4536 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbd18f243cb4fe7944e05d8a85bf79dbc0955719504f5b476e1b0af83bad6c65"
Oct 30 22:36:11.155049 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.155033    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:36:11.243475 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243443    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs\" (UniqueName: \"kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs\") pod \"5799f173-0882-4895-8049-087dfac11054\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") "
Oct 30 22:36:11.243656 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243513    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home\") pod \"5799f173-0882-4895-8049-087dfac11054\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") "
Oct 30 22:36:11.243656 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243550    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir\") pod \"5799f173-0882-4895-8049-087dfac11054\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") "
Oct 30 22:36:11.243656 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243569    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache\") pod \"5799f173-0882-4895-8049-087dfac11054\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") "
Oct 30 22:36:11.243656 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243586    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm\") pod \"5799f173-0882-4895-8049-087dfac11054\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") "
Oct 30 22:36:11.243656 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243645    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt\" (UniqueName: \"kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt\") pod \"5799f173-0882-4895-8049-087dfac11054\" (UID: \"5799f173-0882-4895-8049-087dfac11054\") "
Oct 30 22:36:11.243950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243810    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home" pod "5799f173-0882-4895-8049-087dfac11054" (UID: "5799f173-0882-4895-8049-087dfac11054"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:11.243950 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243870    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache" pod "5799f173-0882-4895-8049-087dfac11054" (UID: "5799f173-0882-4895-8049-087dfac11054"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:11.244026 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.243966    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir" pod "5799f173-0882-4895-8049-087dfac11054" (UID: "5799f173-0882-4895-8049-087dfac11054"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:11.244456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.244351    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:11.244456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.244423    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:11.244456 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.244442    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:11.246089 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.246061    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm" pod "5799f173-0882-4895-8049-087dfac11054" (UID: "5799f173-0882-4895-8049-087dfac11054"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:11.246205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.246127    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt" pod "5799f173-0882-4895-8049-087dfac11054" (UID: "5799f173-0882-4895-8049-087dfac11054"). InnerVolumeSpecName "kube-api-access-kllgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:11.246205 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.246131    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs" pod "5799f173-0882-4895-8049-087dfac11054" (UID: "5799f173-0882-4895-8049-087dfac11054"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:11.346117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.346077    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5799f173-0882-4895-8049-087dfac11054-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:11.346117 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.346116    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kllgt\" (UniqueName: \"kubernetes.io/projected/5799f173-0882-4895-8049-087dfac11054-kube-api-access-kllgt\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:11.346306 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:11.346133    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5799f173-0882-4895-8049-087dfac11054-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:12.165962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:12.165920    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"
Oct 30 22:36:12.333290 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:12.333249    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"]
Oct 30 22:36:12.382826 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:12.382793    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2a4123/simulator-w0-2a4123-kserve-55fdf675dd-thxzg"]
Oct 30 22:36:13.164068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:13.164035    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5799f173-0882-4895-8049-087dfac11054" path="/var/lib/kubelet/pods/5799f173-0882-4895-8049-087dfac11054/volumes"
Oct 30 22:36:15.789589 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:15.789556    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"]
Oct 30 22:36:20.861952 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:36:20.861837    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:36:30.481906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:30.481877    4536 scope.go:122] "RemoveContainer" containerID="7f8834e931b5687c10122fa3b1b54117ff07eaf8ecf94d5bd32ba6e8ab461467"
Oct 30 22:36:30.810735 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:30.810636    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main" containerID="cri-o://9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1" gracePeriod=45
Oct 30 22:36:31.087114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.087092    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:36:31.185361 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185334    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs\") pod \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") "
Oct 30 22:36:31.185538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185383    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc\" (UniqueName: \"kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc\") pod \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") "
Oct 30 22:36:31.185538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185465    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home\") pod \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") "
Oct 30 22:36:31.185630 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185536    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm\") pod \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") "
Oct 30 22:36:31.185680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185643    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir\") pod \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") "
Oct 30 22:36:31.185738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185687    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache\") pod \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\" (UID: \"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb\") "
Oct 30 22:36:31.185805 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185774    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home" pod "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" (UID: "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:31.185944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.185914    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir" pod "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" (UID: "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:31.186024 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.186001    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache" pod "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" (UID: "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:31.186124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.186104    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:31.186124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.186122    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:31.186124 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.186136    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:31.187719 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.187693    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs" pod "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" (UID: "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:31.187825 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.187809    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm" pod "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" (UID: "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:31.187887 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.187870    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc" pod "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" (UID: "4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb"). InnerVolumeSpecName "kube-api-access-sxpmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:31.287013 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.286979    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:31.287013 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.287015    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxpmc\" (UniqueName: \"kubernetes.io/projected/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-kube-api-access-sxpmc\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:31.287216 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.287032    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:31.303339 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.303301    4536 generic.go:356] "Generic (PLEG): container finished" podID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerID="9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1" exitCode=0
Oct 30 22:36:31.303533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.303346    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"
Oct 30 22:36:31.303533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.303381    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" event={"ID":"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb","Type":"ContainerDied","Data":"9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1"}
Oct 30 22:36:31.303533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.303443    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr" event={"ID":"4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb","Type":"ContainerDied","Data":"d3ef4d5888687f54d96b4f8b12989c4c7dd677e729c05b71b63f0f2a55946169"}
Oct 30 22:36:31.303533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.303466    4536 scope.go:122] "RemoveContainer" containerID="9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1"
Oct 30 22:36:31.313643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.313628    4536 scope.go:122] "RemoveContainer" containerID="9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1"
Oct 30 22:36:31.313876 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:36:31.313854    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1\": container with ID starting with 9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1 not found: ID does not exist" containerID="9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1"
Oct 30 22:36:31.313939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.313888    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1"} err="failed to get container status \"9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1\": rpc error: code = NotFound desc = could not find container \"9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1\": container with ID starting with 9d54086dde67528340a1c0b8a334766f2ee4382f529f1f71d2ca707652924ef1 not found: ID does not exist"
Oct 30 22:36:31.471493 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.471458    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"]
Oct 30 22:36:31.480272 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:31.480238    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-97aa64/premium-w4-97aa64-kserve-6565d769b6-f8vzr"]
Oct 30 22:36:33.163725 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:33.163692    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" path="/var/lib/kubelet/pods/4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb/volumes"
Oct 30 22:36:50.805296 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:36:50.805258    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:36:58.262275 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:58.262246    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"]
Oct 30 22:36:58.262685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:36:58.262480    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy" containerID="cri-o://12174d3961463f4f1947241d4e386035dbad627ff1d31512403c22670bd3a705" gracePeriod=30
Oct 30 22:37:03.475388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.475355    4536 generic.go:356] "Generic (PLEG): container finished" podID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerID="12174d3961463f4f1947241d4e386035dbad627ff1d31512403c22670bd3a705" exitCode=0
Oct 30 22:37:03.475834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.475429    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" event={"ID":"744e95b0-6cc9-4160-898e-a7aab48c54d3","Type":"ContainerDied","Data":"12174d3961463f4f1947241d4e386035dbad627ff1d31512403c22670bd3a705"}
Oct 30 22:37:03.515221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.515189    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:37:03.594574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594493    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594530    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594574 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594564    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594599    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594639    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594683    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594736    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.594834 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594776    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.595103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594844    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo\") pod \"744e95b0-6cc9-4160-898e-a7aab48c54d3\" (UID: \"744e95b0-6cc9-4160-898e-a7aab48c54d3\") "
Oct 30 22:37:03.595103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.594949    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:03.595103 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595088    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:03.595277 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595106    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:03.595277 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595163    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:03.595388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595275    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:03.595388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595278    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e95b0-6cc9-4160-898e-a7aab48c54d3-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.595388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595315    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.595388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595331    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.595388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.595346    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.597122 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.597098    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:37:03.597225 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.597135    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "kube-api-access-wd7lt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:03.597428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.597373    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:03.597428 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.597375    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token" pod "744e95b0-6cc9-4160-898e-a7aab48c54d3" (UID: "744e95b0-6cc9-4160-898e-a7aab48c54d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:03.696925 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.696891    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.696925 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.696921    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.696925 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.696930    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.697217 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.696939    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd7lt\" (UniqueName: \"kubernetes.io/projected/744e95b0-6cc9-4160-898e-a7aab48c54d3-kube-api-access-wd7lt\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:03.697217 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:03.696949    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e95b0-6cc9-4160-898e-a7aab48c54d3-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:04.482439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:04.482407    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"
Oct 30 22:37:04.482439 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:04.482424    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv" event={"ID":"744e95b0-6cc9-4160-898e-a7aab48c54d3","Type":"ContainerDied","Data":"42aa0f475ebfbe483d069d753ce6ea87fd8cdfe1f69d9aed82eb56ff3f8db61c"}
Oct 30 22:37:04.482949 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:04.482491    4536 scope.go:122] "RemoveContainer" containerID="12174d3961463f4f1947241d4e386035dbad627ff1d31512403c22670bd3a705"
Oct 30 22:37:04.706987 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:04.706952    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"]
Oct 30 22:37:04.712485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:04.712454    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2a4123-openshift-default-554c56cc99-nztgv"]
Oct 30 22:37:05.164253 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:05.164221    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" path="/var/lib/kubelet/pods/744e95b0-6cc9-4160-898e-a7aab48c54d3/volumes"
Oct 30 22:37:06.161369 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.161334    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"]
Oct 30 22:37:06.161859 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.161548    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" podUID="87f414d1-0414-4945-8552-b2ac20bf4649" containerName="maas-api" containerID="cri-o://d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d" gracePeriod=30
Oct 30 22:37:06.472175 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.472144    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:37:06.496539 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.496503    4536 generic.go:356] "Generic (PLEG): container finished" podID="87f414d1-0414-4945-8552-b2ac20bf4649" containerID="d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d" exitCode=0
Oct 30 22:37:06.496700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.496575    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"
Oct 30 22:37:06.496700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.496593    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" event={"ID":"87f414d1-0414-4945-8552-b2ac20bf4649","Type":"ContainerDied","Data":"d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d"}
Oct 30 22:37:06.496700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.496664    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2" event={"ID":"87f414d1-0414-4945-8552-b2ac20bf4649","Type":"ContainerDied","Data":"9b3fc9453d3a58e8c073304133b6e1d5949873651abf027526be43d3349f584d"}
Oct 30 22:37:06.496700 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.496695    4536 scope.go:122] "RemoveContainer" containerID="d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d"
Oct 30 22:37:06.507555 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.507539    4536 scope.go:122] "RemoveContainer" containerID="d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d"
Oct 30 22:37:06.507846 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:37:06.507829    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d\": container with ID starting with d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d not found: ID does not exist" containerID="d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d"
Oct 30 22:37:06.507939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.507855    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d"} err="failed to get container status \"d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d\": rpc error: code = NotFound desc = could not find container \"d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d\": container with ID starting with d9350d949dbeb93b638478a976e391815fbc969713b4214d0b250889b7da9a3d not found: ID does not exist"
Oct 30 22:37:06.532276 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.532255    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca\") pod \"87f414d1-0414-4945-8552-b2ac20bf4649\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") "
Oct 30 22:37:06.532430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.532310    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75\" (UniqueName: \"kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75\") pod \"87f414d1-0414-4945-8552-b2ac20bf4649\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") "
Oct 30 22:37:06.532430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.532348    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\") pod \"87f414d1-0414-4945-8552-b2ac20bf4649\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") "
Oct 30 22:37:06.532430 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.532428    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs\") pod \"87f414d1-0414-4945-8552-b2ac20bf4649\" (UID: \"87f414d1-0414-4945-8552-b2ac20bf4649\") "
Oct 30 22:37:06.532674 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.532649    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca" pod "87f414d1-0414-4945-8552-b2ac20bf4649" (UID: "87f414d1-0414-4945-8552-b2ac20bf4649"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:06.532767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.532744    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87f414d1-0414-4945-8552-b2ac20bf4649-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.534610 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.534586    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls" pod "87f414d1-0414-4945-8552-b2ac20bf4649" (UID: "87f414d1-0414-4945-8552-b2ac20bf4649"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:06.534698 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.534647    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs" pod "87f414d1-0414-4945-8552-b2ac20bf4649" (UID: "87f414d1-0414-4945-8552-b2ac20bf4649"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:06.534767 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.534746    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75" pod "87f414d1-0414-4945-8552-b2ac20bf4649" (UID: "87f414d1-0414-4945-8552-b2ac20bf4649"). InnerVolumeSpecName "kube-api-access-ltz75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:06.633635 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.633601    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltz75\" (UniqueName: \"kubernetes.io/projected/87f414d1-0414-4945-8552-b2ac20bf4649-kube-api-access-ltz75\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.633635 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.633630    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.633635 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.633640    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f414d1-0414-4945-8552-b2ac20bf4649-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:06.919387 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.919344    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"]
Oct 30 22:37:06.926460 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:06.926434    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2a4123-7857bc788d-sctw2"]
Oct 30 22:37:07.183310 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:07.183195    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87f414d1-0414-4945-8552-b2ac20bf4649" path="/var/lib/kubelet/pods/87f414d1-0414-4945-8552-b2ac20bf4649/volumes"
Oct 30 22:37:20.805109 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:37:20.805020    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:37:22.910745 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:22.910709    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"]
Oct 30 22:37:22.911123 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:22.910923    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" podUID="1b3d1f77-b03e-4553-9331-75bffb1d391a" containerName="maas-api" containerID="cri-o://0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856" gracePeriod=30
Oct 30 22:37:23.207471 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.202162    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:37:23.333114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.333073    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg\" (UniqueName: \"kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg\") pod \"1b3d1f77-b03e-4553-9331-75bffb1d391a\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") "
Oct 30 22:37:23.333522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.333497    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca\") pod \"1b3d1f77-b03e-4553-9331-75bffb1d391a\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") "
Oct 30 22:37:23.333679 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.333662    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs\") pod \"1b3d1f77-b03e-4553-9331-75bffb1d391a\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") "
Oct 30 22:37:23.334291 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.334271    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\") pod \"1b3d1f77-b03e-4553-9331-75bffb1d391a\" (UID: \"1b3d1f77-b03e-4553-9331-75bffb1d391a\") "
Oct 30 22:37:23.338004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.337900    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca" pod "1b3d1f77-b03e-4553-9331-75bffb1d391a" (UID: "1b3d1f77-b03e-4553-9331-75bffb1d391a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:23.339880 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.339836    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls" pod "1b3d1f77-b03e-4553-9331-75bffb1d391a" (UID: "1b3d1f77-b03e-4553-9331-75bffb1d391a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:23.364443 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.361874    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg" pod "1b3d1f77-b03e-4553-9331-75bffb1d391a" (UID: "1b3d1f77-b03e-4553-9331-75bffb1d391a"). InnerVolumeSpecName "kube-api-access-xpdvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:23.365072 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.365043    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs" pod "1b3d1f77-b03e-4553-9331-75bffb1d391a" (UID: "1b3d1f77-b03e-4553-9331-75bffb1d391a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:23.435388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.435354    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpdvg\" (UniqueName: \"kubernetes.io/projected/1b3d1f77-b03e-4553-9331-75bffb1d391a-kube-api-access-xpdvg\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:23.435388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.435382    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b3d1f77-b03e-4553-9331-75bffb1d391a-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:23.435388 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.435406    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:23.435612 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.435418    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b3d1f77-b03e-4553-9331-75bffb1d391a-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:23.614942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.614864    4536 generic.go:356] "Generic (PLEG): container finished" podID="1b3d1f77-b03e-4553-9331-75bffb1d391a" containerID="0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856" exitCode=0
Oct 30 22:37:23.614942 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.614913    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"
Oct 30 22:37:23.615128 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.614947    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" event={"ID":"1b3d1f77-b03e-4553-9331-75bffb1d391a","Type":"ContainerDied","Data":"0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856"}
Oct 30 22:37:23.615128 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.615002    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv" event={"ID":"1b3d1f77-b03e-4553-9331-75bffb1d391a","Type":"ContainerDied","Data":"ca4720f1ab2bb076e101260949f830a726d0f23d54a5491cfb48b53ec9dc48bf"}
Oct 30 22:37:23.615128 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.615024    4536 scope.go:122] "RemoveContainer" containerID="0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856"
Oct 30 22:37:23.624340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.624319    4536 scope.go:122] "RemoveContainer" containerID="0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856"
Oct 30 22:37:23.624628 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:37:23.624610    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856\": container with ID starting with 0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856 not found: ID does not exist" containerID="0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856"
Oct 30 22:37:23.624702 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.624635    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856"} err="failed to get container status \"0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856\": rpc error: code = NotFound desc = could not find container \"0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856\": container with ID starting with 0e80edc3c8f408d0eb4f5fbf23c7e588ba9fac4bee1a66780e3ee5a24ff56856 not found: ID does not exist"
Oct 30 22:37:23.782150 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.782115    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"]
Oct 30 22:37:23.787617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:23.787592    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-97aa64-67b4d58f8f-qh9vv"]
Oct 30 22:37:25.164457 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:25.164427    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b3d1f77-b03e-4553-9331-75bffb1d391a" path="/var/lib/kubelet/pods/1b3d1f77-b03e-4553-9331-75bffb1d391a/volumes"
Oct 30 22:37:33.501965 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:33.501933    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"]
Oct 30 22:37:33.502444 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:33.502170    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy" containerID="cri-o://af02960b9e3685cd58c7f57855804265f67230866c4fedd0bf64304c537d2b87" gracePeriod=30
Oct 30 22:37:38.701774 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.701738    4536 generic.go:356] "Generic (PLEG): container finished" podID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerID="af02960b9e3685cd58c7f57855804265f67230866c4fedd0bf64304c537d2b87" exitCode=0
Oct 30 22:37:38.702510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.701819    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" event={"ID":"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005","Type":"ContainerDied","Data":"af02960b9e3685cd58c7f57855804265f67230866c4fedd0bf64304c537d2b87"}
Oct 30 22:37:38.759632 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.759607    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:37:38.801536 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801450    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801541    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801582    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801633    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801666    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801760    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801845    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801876    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.801962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.801938    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data\") pod \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\" (UID: \"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005\") "
Oct 30 22:37:38.802174 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.802025    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:38.802174 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.802033    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:38.802708 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.802326    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:38.802708 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.802346    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.802708 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.802370    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.802708 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.802580    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:38.803194 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.803112    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:38.804463 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.804389    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "kube-api-access-shbqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:38.804650 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.804515    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:37:38.805035 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.805003    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:38.805342 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.805295    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy" pod "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" (UID: "5c9a5a29-7dcf-45c7-a03b-8ae4fc906005"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:38.903095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903058    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.903095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903090    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.903095 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903099    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.903330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903107    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.903330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903116    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.903330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903124    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:38.903330 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:38.903132    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shbqv\" (UniqueName: \"kubernetes.io/projected/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005-kube-api-access-shbqv\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:39.708043 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:39.708013    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"
Oct 30 22:37:39.708579 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:39.708038    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq" event={"ID":"5c9a5a29-7dcf-45c7-a03b-8ae4fc906005","Type":"ContainerDied","Data":"a246a2a2ba0c0d2625d5ebaf213f2c54fb89d24e8748e1870476f34f61f56a4c"}
Oct 30 22:37:39.708579 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:39.708106    4536 scope.go:122] "RemoveContainer" containerID="af02960b9e3685cd58c7f57855804265f67230866c4fedd0bf64304c537d2b87"
Oct 30 22:37:39.925553 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:39.925522    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"]
Oct 30 22:37:39.975835 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:39.975758    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-97aa64-openshift-default-569955745-qzbmq"]
Oct 30 22:37:41.164009 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:37:41.163981    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" path="/var/lib/kubelet/pods/5c9a5a29-7dcf-45c7-a03b-8ae4fc906005/volumes"
Oct 30 22:37:50.804804 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:37:50.804761    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:38:20.805416 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:38:20.805355    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:38:50.804777 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:38:50.804695    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:39:20.805233 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:39:20.805189    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:39:50.805963 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:39:50.805922    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:40:04.791737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:04.791695    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"]
Oct 30 22:40:04.792336 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:04.791971    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" containerID="cri-o://42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74" gracePeriod=30
Oct 30 22:40:07.861195 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:07.861147    4536 patch_prober.go:29] interesting pod/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:40:07.861678 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:07.861228    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:40:10.052236 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.052209    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:40:10.108633 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108570    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.108633 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108612    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.108889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108654    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.108889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108723    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.108889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108789    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.108889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108828    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.108889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108850    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:10.108889 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108863    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.109212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108950    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.109212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.108996    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data\") pod \"740b2afb-173f-4c0e-a22e-892cb69da226\" (UID: \"740b2afb-173f-4c0e-a22e-892cb69da226\") "
Oct 30 22:40:10.109212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109057    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:10.109212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109174    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:10.109212 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109190    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:10.109528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109368    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:10.109528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109382    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.109528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109423    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.109528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109439    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/740b2afb-173f-4c0e-a22e-892cb69da226-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.109528 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.109500    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.111104 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.111073    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:40:10.111104 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.111091    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:10.111279 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.111157    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:10.111338 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.111320    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv" pod "740b2afb-173f-4c0e-a22e-892cb69da226" (UID: "740b2afb-173f-4c0e-a22e-892cb69da226"). InnerVolumeSpecName "kube-api-access-qtwxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:10.210597 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.210560    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/740b2afb-173f-4c0e-a22e-892cb69da226-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.210597 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.210600    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.210798 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.210616    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtwxv\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-kube-api-access-qtwxv\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.210798 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.210632    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/740b2afb-173f-4c0e-a22e-892cb69da226-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.210798 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.210645    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/740b2afb-173f-4c0e-a22e-892cb69da226-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:10.435485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.435360    4536 generic.go:356] "Generic (PLEG): container finished" podID="740b2afb-173f-4c0e-a22e-892cb69da226" containerID="42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74" exitCode=0
Oct 30 22:40:10.435485 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.435441    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"
Oct 30 22:40:10.435692 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.435439    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" event={"ID":"740b2afb-173f-4c0e-a22e-892cb69da226","Type":"ContainerDied","Data":"42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74"}
Oct 30 22:40:10.435692 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.435560    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf" event={"ID":"740b2afb-173f-4c0e-a22e-892cb69da226","Type":"ContainerDied","Data":"8d60479b0e60864afff8ebef5f586bbb202510763a6aa81263372861ace9b49a"}
Oct 30 22:40:10.435692 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.435590    4536 scope.go:122] "RemoveContainer" containerID="42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74"
Oct 30 22:40:10.445706 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.445663    4536 scope.go:122] "RemoveContainer" containerID="42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74"
Oct 30 22:40:10.445989 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:10.445961    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74\": container with ID starting with 42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74 not found: ID does not exist" containerID="42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74"
Oct 30 22:40:10.446060 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.445991    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74"} err="failed to get container status \"42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74\": rpc error: code = NotFound desc = could not find container \"42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74\": container with ID starting with 42d1d823560f46313787dcdadf91b2683b0f4e20e6aecb06e017f8f5f3f02c74 not found: ID does not exist"
Oct 30 22:40:10.603761 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.603717    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"]
Oct 30 22:40:10.611053 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:10.611027    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e54161-openshift-default-7899496757-gwjzf"]
Oct 30 22:40:11.164232 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:11.164199    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" path="/var/lib/kubelet/pods/740b2afb-173f-4c0e-a22e-892cb69da226/volumes"
Oct 30 22:40:16.456157 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.456118    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"]
Oct 30 22:40:16.456633 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.456313    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" podUID="e3ded310-d258-4da0-81e5-dc7924ca1886" containerName="maas-api" containerID="cri-o://1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5" gracePeriod=30
Oct 30 22:40:16.762122 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.762097    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:40:16.882851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.882817    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\") pod \"e3ded310-d258-4da0-81e5-dc7924ca1886\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") "
Oct 30 22:40:16.883057 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.882870    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl\" (UniqueName: \"kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl\") pod \"e3ded310-d258-4da0-81e5-dc7924ca1886\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") "
Oct 30 22:40:16.883057 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.882912    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs\") pod \"e3ded310-d258-4da0-81e5-dc7924ca1886\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") "
Oct 30 22:40:16.883193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.883158    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca\") pod \"e3ded310-d258-4da0-81e5-dc7924ca1886\" (UID: \"e3ded310-d258-4da0-81e5-dc7924ca1886\") "
Oct 30 22:40:16.883900 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.883868    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca" pod "e3ded310-d258-4da0-81e5-dc7924ca1886" (UID: "e3ded310-d258-4da0-81e5-dc7924ca1886"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:16.885885 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.885845    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls" pod "e3ded310-d258-4da0-81e5-dc7924ca1886" (UID: "e3ded310-d258-4da0-81e5-dc7924ca1886"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:16.886234 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.886212    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl" pod "e3ded310-d258-4da0-81e5-dc7924ca1886" (UID: "e3ded310-d258-4da0-81e5-dc7924ca1886"). InnerVolumeSpecName "kube-api-access-r8rxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:16.886515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.886491    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs" pod "e3ded310-d258-4da0-81e5-dc7924ca1886" (UID: "e3ded310-d258-4da0-81e5-dc7924ca1886"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:16.943221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.943186    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"]
Oct 30 22:40:16.985040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.984964    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8rxl\" (UniqueName: \"kubernetes.io/projected/e3ded310-d258-4da0-81e5-dc7924ca1886-kube-api-access-r8rxl\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:16.985040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.984991    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:16.985040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.985001    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ded310-d258-4da0-81e5-dc7924ca1886-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:16.985040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.985032    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3ded310-d258-4da0-81e5-dc7924ca1886-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:16.999939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:16.999910    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"]
Oct 30 22:40:17.479516 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.479477    4536 generic.go:356] "Generic (PLEG): container finished" podID="e3ded310-d258-4da0-81e5-dc7924ca1886" containerID="1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5" exitCode=0
Oct 30 22:40:17.480114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.479531    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" event={"ID":"e3ded310-d258-4da0-81e5-dc7924ca1886","Type":"ContainerDied","Data":"1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5"}
Oct 30 22:40:17.480114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.479581    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp" event={"ID":"e3ded310-d258-4da0-81e5-dc7924ca1886","Type":"ContainerDied","Data":"0bd7582599089ecf3e36e72fe6da14c70376146911f634ff981369be7d62becd"}
Oct 30 22:40:17.480114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.479609    4536 scope.go:122] "RemoveContainer" containerID="1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5"
Oct 30 22:40:17.480114 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.479769    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"
Oct 30 22:40:17.505449 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.503972    4536 scope.go:122] "RemoveContainer" containerID="1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5"
Oct 30 22:40:17.516471 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:17.516377    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5\": container with ID starting with 1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5 not found: ID does not exist" containerID="1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5"
Oct 30 22:40:17.516471 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.516437    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5"} err="failed to get container status \"1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5\": rpc error: code = NotFound desc = could not find container \"1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5\": container with ID starting with 1b25e6241f33cd618c3323a7bc605ef0ec6bc382309ae1e83c657698d71109c5 not found: ID does not exist"
Oct 30 22:40:17.651917 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.651869    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"]
Oct 30 22:40:17.659759 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:17.659703    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-d50b13-76d48b79bc-8sdzp"]
Oct 30 22:40:19.163906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:19.163876    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3ded310-d258-4da0-81e5-dc7924ca1886" path="/var/lib/kubelet/pods/e3ded310-d258-4da0-81e5-dc7924ca1886/volumes"
Oct 30 22:40:20.805101 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:20.805063    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:40:21.943827 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:21.943785    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn" podUID="de107361-6bf7-462d-abd8-3ef49de8e097" containerName="payload-processing" containerID="cri-o://8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967" gracePeriod=25
Oct 30 22:40:22.000359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.000303    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742" podUID="657220e1-3d93-4875-8a4c-b727c3e9751e" containerName="payload-pre-processing" containerID="cri-o://aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1" gracePeriod=25
Oct 30 22:40:22.262742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.262709    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:40:22.278877 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.278856    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:40:22.366747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.365732    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh\" (UniqueName: \"kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh\") pod \"de107361-6bf7-462d-abd8-3ef49de8e097\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") "
Oct 30 22:40:22.366747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.365789    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume\") pod \"de107361-6bf7-462d-abd8-3ef49de8e097\" (UID: \"de107361-6bf7-462d-abd8-3ef49de8e097\") "
Oct 30 22:40:22.369489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.369438    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume" pod "de107361-6bf7-462d-abd8-3ef49de8e097" (UID: "de107361-6bf7-462d-abd8-3ef49de8e097"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:22.379416 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.379365    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh" pod "de107361-6bf7-462d-abd8-3ef49de8e097" (UID: "de107361-6bf7-462d-abd8-3ef49de8e097"). InnerVolumeSpecName "kube-api-access-9xsgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:22.467199 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.467119    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw\" (UniqueName: \"kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw\") pod \"657220e1-3d93-4875-8a4c-b727c3e9751e\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") "
Oct 30 22:40:22.467377 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.467222    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume\") pod \"657220e1-3d93-4875-8a4c-b727c3e9751e\" (UID: \"657220e1-3d93-4875-8a4c-b727c3e9751e\") "
Oct 30 22:40:22.467581 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.467560    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xsgh\" (UniqueName: \"kubernetes.io/projected/de107361-6bf7-462d-abd8-3ef49de8e097-kube-api-access-9xsgh\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:22.467641 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.467586    4536 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de107361-6bf7-462d-abd8-3ef49de8e097-plugins-config-volume\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:22.467641 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.467597    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume" pod "657220e1-3d93-4875-8a4c-b727c3e9751e" (UID: "657220e1-3d93-4875-8a4c-b727c3e9751e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:22.469272 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.469253    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw" pod "657220e1-3d93-4875-8a4c-b727c3e9751e" (UID: "657220e1-3d93-4875-8a4c-b727c3e9751e"). InnerVolumeSpecName "kube-api-access-686xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:22.514742 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.514712    4536 generic.go:356] "Generic (PLEG): container finished" podID="de107361-6bf7-462d-abd8-3ef49de8e097" containerID="8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967" exitCode=0
Oct 30 22:40:22.514882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.514759    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"
Oct 30 22:40:22.514882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.514802    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn" event={"ID":"de107361-6bf7-462d-abd8-3ef49de8e097","Type":"ContainerDied","Data":"8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967"}
Oct 30 22:40:22.514882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.514879    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn" event={"ID":"de107361-6bf7-462d-abd8-3ef49de8e097","Type":"ContainerDied","Data":"b7080ea4ff7c0ed53cfb9639fbd9cafde83be3947cda903c9bcaaea30a1d9b0a"}
Oct 30 22:40:22.515084 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.514910    4536 scope.go:122] "RemoveContainer" containerID="8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967"
Oct 30 22:40:22.516243 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.516218    4536 generic.go:356] "Generic (PLEG): container finished" podID="657220e1-3d93-4875-8a4c-b727c3e9751e" containerID="aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1" exitCode=0
Oct 30 22:40:22.516360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.516259    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"
Oct 30 22:40:22.516360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.516320    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742" event={"ID":"657220e1-3d93-4875-8a4c-b727c3e9751e","Type":"ContainerDied","Data":"aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1"}
Oct 30 22:40:22.516360 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.516351    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742" event={"ID":"657220e1-3d93-4875-8a4c-b727c3e9751e","Type":"ContainerDied","Data":"be66c5b3440402eb853aef749368ea7ac20adfcf929724459f6f5125bcd59264"}
Oct 30 22:40:22.526533 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.526514    4536 scope.go:122] "RemoveContainer" containerID="8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967"
Oct 30 22:40:22.526887 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:22.526863    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967\": container with ID starting with 8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967 not found: ID does not exist" containerID="8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967"
Oct 30 22:40:22.526989 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.526897    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967"} err="failed to get container status \"8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967\": rpc error: code = NotFound desc = could not find container \"8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967\": container with ID starting with 8532fb0bb37f74cae5df79cbdf3992bfb94f3b4c431ff3e32be4bd5b201e0967 not found: ID does not exist"
Oct 30 22:40:22.526989 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.526920    4536 scope.go:122] "RemoveContainer" containerID="aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1"
Oct 30 22:40:22.535802 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.535777    4536 scope.go:122] "RemoveContainer" containerID="aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1"
Oct 30 22:40:22.536083 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:22.536059    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1\": container with ID starting with aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1 not found: ID does not exist" containerID="aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1"
Oct 30 22:40:22.536171 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.536090    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1"} err="failed to get container status \"aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1\": rpc error: code = NotFound desc = could not find container \"aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1\": container with ID starting with aa361400e1933ce4835feb2acb535c376bd3f5f20dd1ab1a529a681ebc8487b1 not found: ID does not exist"
Oct 30 22:40:22.568873 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.568837    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-686xw\" (UniqueName: \"kubernetes.io/projected/657220e1-3d93-4875-8a4c-b727c3e9751e-kube-api-access-686xw\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:22.568873 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.568870    4536 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/657220e1-3d93-4875-8a4c-b727c3e9751e-plugins-config-volume\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:22.751499 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.751384    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"]
Oct 30 22:40:22.757075 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.757052    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d50b13-78878f6b79-ch742"]
Oct 30 22:40:22.870588 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.870559    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"]
Oct 30 22:40:22.885880 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:22.885848    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-d50b13-5dc5fb8b7-2x4dn"]
Oct 30 22:40:23.165086 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:23.165009    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="657220e1-3d93-4875-8a4c-b727c3e9751e" path="/var/lib/kubelet/pods/657220e1-3d93-4875-8a4c-b727c3e9751e/volumes"
Oct 30 22:40:23.165465 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:23.165378    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de107361-6bf7-462d-abd8-3ef49de8e097" path="/var/lib/kubelet/pods/de107361-6bf7-462d-abd8-3ef49de8e097/volumes"
Oct 30 22:40:30.161651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:30.161614    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"]
Oct 30 22:40:30.162081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:30.161820    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy" containerID="cri-o://04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31" gracePeriod=30
Oct 30 22:40:35.408888 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.408861    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:40:35.501956 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.501930    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.501983    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502033    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502059    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502098 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502084    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502331 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502109    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502331 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502135    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502331 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502167    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502331 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502216    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:35.502331 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502235    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket\") pod \"6a8b6174-67fc-4776-bc8f-4a72565638ff\" (UID: \"6a8b6174-67fc-4776-bc8f-4a72565638ff\") "
Oct 30 22:40:35.502621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502474    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:35.502621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502493    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:35.502730 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502596    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:35.502833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502776    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.502833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502798    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a8b6174-67fc-4776-bc8f-4a72565638ff-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.502833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502813    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.502833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502848    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.503162 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.502886    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:35.504538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.504513    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:35.504636 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.504585    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "kube-api-access-knk98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:35.504905 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.504879    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:40:35.504905 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.504884    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy" pod "6a8b6174-67fc-4776-bc8f-4a72565638ff" (UID: "6a8b6174-67fc-4776-bc8f-4a72565638ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:35.584379 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.584343    4536 generic.go:356] "Generic (PLEG): container finished" podID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerID="04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31" exitCode=0
Oct 30 22:40:35.584617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.584407    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"
Oct 30 22:40:35.584617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.584450    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" event={"ID":"6a8b6174-67fc-4776-bc8f-4a72565638ff","Type":"ContainerDied","Data":"04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31"}
Oct 30 22:40:35.584617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.584537    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr" event={"ID":"6a8b6174-67fc-4776-bc8f-4a72565638ff","Type":"ContainerDied","Data":"f1dcf883ffe84194e793c8da95fbd393f0f258a0de02cfff3a2e3d06adb479c7"}
Oct 30 22:40:35.584617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.584589    4536 scope.go:122] "RemoveContainer" containerID="04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31"
Oct 30 22:40:35.595078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.595062    4536 scope.go:122] "RemoveContainer" containerID="ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"
Oct 30 22:40:35.603315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603296    4536 scope.go:122] "RemoveContainer" containerID="04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31"
Oct 30 22:40:35.603585 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:35.603567    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31\": container with ID starting with 04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31 not found: ID does not exist" containerID="04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31"
Oct 30 22:40:35.603654 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603594    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31"} err="failed to get container status \"04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31\": rpc error: code = NotFound desc = could not find container \"04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31\": container with ID starting with 04708266d5a335d76d8e941b64c4b55bfa498878158259d4feed143f8400be31 not found: ID does not exist"
Oct 30 22:40:35.603654 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603625    4536 scope.go:122] "RemoveContainer" containerID="ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"
Oct 30 22:40:35.603814 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603794    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.603872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603817    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.603872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603828    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.603872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603836    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a8b6174-67fc-4776-bc8f-4a72565638ff-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.603872 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603850    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knk98\" (UniqueName: \"kubernetes.io/projected/6a8b6174-67fc-4776-bc8f-4a72565638ff-kube-api-access-knk98\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:35.603994 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:35.603885    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa\": container with ID starting with ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa not found: ID does not exist" containerID="ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"
Oct 30 22:40:35.603994 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.603904    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa"} err="failed to get container status \"ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa\": rpc error: code = NotFound desc = could not find container \"ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa\": container with ID starting with ade756cd07dad0f3486542f625c2985219e926172b50823472ed7bf2b30c28fa not found: ID does not exist"
Oct 30 22:40:35.753838 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.753725    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"]
Oct 30 22:40:35.761280 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:35.761251    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-d50b13-openshift-default-d9f596d45-fjcsr"]
Oct 30 22:40:37.163539 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:40:37.163508    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" path="/var/lib/kubelet/pods/6a8b6174-67fc-4776-bc8f-4a72565638ff/volumes"
Oct 30 22:40:50.805010 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:40:50.804970    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:41:05.333544 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:05.333510    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"]
Oct 30 22:41:05.334081 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:05.333730    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy" containerID="cri-o://4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0" gracePeriod=30
Oct 30 22:41:10.594523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.594469    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:41:10.672997 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.672969    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673017    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673038    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673072    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673117    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673161 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673157    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673247    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673258    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:10.673464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673279    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673316    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data\") pod \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\" (UID: \"faef18ca-2ab9-4f4d-9611-7e7b49314f18\") "
Oct 30 22:41:10.673694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673577    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:10.673694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673615    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:10.673694 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673653    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:10.673859 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673749    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:10.674079 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.673999    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.674079 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.674021    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.674079 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.674035    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.674079 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.674049    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.674079 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.674061    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.675510 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.675484    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "kube-api-access-tg5nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:10.675611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.675553    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:41:10.675611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.675594    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:10.675611 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.675594    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy" pod "faef18ca-2ab9-4f4d-9611-7e7b49314f18" (UID: "faef18ca-2ab9-4f4d-9611-7e7b49314f18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:10.763747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.763712    4536 generic.go:356] "Generic (PLEG): container finished" podID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerID="4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0" exitCode=0
Oct 30 22:41:10.763924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.763801    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"
Oct 30 22:41:10.763924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.763799    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" event={"ID":"faef18ca-2ab9-4f4d-9611-7e7b49314f18","Type":"ContainerDied","Data":"4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0"}
Oct 30 22:41:10.763924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.763862    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2" event={"ID":"faef18ca-2ab9-4f4d-9611-7e7b49314f18","Type":"ContainerDied","Data":"72247176b9d3149cfa768097d0dbf44a69953567cbda4317686f93d781d534ac"}
Oct 30 22:41:10.763924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.763885    4536 scope.go:122] "RemoveContainer" containerID="4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0"
Oct 30 22:41:10.774910 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.774881    4536 scope.go:122] "RemoveContainer" containerID="26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"
Oct 30 22:41:10.775289 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.775263    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg5nl\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-kube-api-access-tg5nl\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.775432 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.775295    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.775432 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.775314    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.775432 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.775329    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/faef18ca-2ab9-4f4d-9611-7e7b49314f18-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:10.783466 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.783444    4536 scope.go:122] "RemoveContainer" containerID="4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0"
Oct 30 22:41:10.783787 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:10.783769    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0\": container with ID starting with 4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0 not found: ID does not exist" containerID="4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0"
Oct 30 22:41:10.783863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.783801    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0"} err="failed to get container status \"4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0\": rpc error: code = NotFound desc = could not find container \"4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0\": container with ID starting with 4ac84055e5e99a1324f15c994fd65b0f39c875bf043958cc4f3f65b5ee039ae0 not found: ID does not exist"
Oct 30 22:41:10.783863 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.783826    4536 scope.go:122] "RemoveContainer" containerID="26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"
Oct 30 22:41:10.784092 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:10.784077    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818\": container with ID starting with 26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818 not found: ID does not exist" containerID="26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"
Oct 30 22:41:10.784145 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.784100    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818"} err="failed to get container status \"26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818\": rpc error: code = NotFound desc = could not find container \"26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818\": container with ID starting with 26ec0d185d06a9796b55b41bb61221d9de52b77017a3a611ba57a0a79b250818 not found: ID does not exist"
Oct 30 22:41:10.819231 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:10.819196    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"]
Oct 30 22:41:11.063536 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:11.063505    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"]
Oct 30 22:41:11.069302 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:11.069276    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-6c7d3f-openshift-default-58dcc8b59-sncg2"]
Oct 30 22:41:11.163150 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:11.163117    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" path="/var/lib/kubelet/pods/faef18ca-2ab9-4f4d-9611-7e7b49314f18/volumes"
Oct 30 22:41:20.804927 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:20.804878    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:41:25.839651 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:25.839599    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main" containerID="cri-o://6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e" gracePeriod=45
Oct 30 22:41:26.090506 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.090444    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:41:26.229913 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.229878    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs\") pod \"a1f76beb-38b7-4318-b025-5669ffaff916\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") "
Oct 30 22:41:26.230102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.229929    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm\") pod \"a1f76beb-38b7-4318-b025-5669ffaff916\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") "
Oct 30 22:41:26.230102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.229949    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache\") pod \"a1f76beb-38b7-4318-b025-5669ffaff916\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") "
Oct 30 22:41:26.230102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230010    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p\" (UniqueName: \"kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p\") pod \"a1f76beb-38b7-4318-b025-5669ffaff916\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") "
Oct 30 22:41:26.230102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230034    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir\") pod \"a1f76beb-38b7-4318-b025-5669ffaff916\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") "
Oct 30 22:41:26.230102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230083    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home\") pod \"a1f76beb-38b7-4318-b025-5669ffaff916\" (UID: \"a1f76beb-38b7-4318-b025-5669ffaff916\") "
Oct 30 22:41:26.230391 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230280    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache" pod "a1f76beb-38b7-4318-b025-5669ffaff916" (UID: "a1f76beb-38b7-4318-b025-5669ffaff916"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:26.230391 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230322    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir" pod "a1f76beb-38b7-4318-b025-5669ffaff916" (UID: "a1f76beb-38b7-4318-b025-5669ffaff916"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:26.231004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230662    4536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-model-cache\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:26.231004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.230686    4536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-tmp-dir\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:26.231167 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.231088    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home" pod "a1f76beb-38b7-4318-b025-5669ffaff916" (UID: "a1f76beb-38b7-4318-b025-5669ffaff916"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:26.232434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.232382    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm" pod "a1f76beb-38b7-4318-b025-5669ffaff916" (UID: "a1f76beb-38b7-4318-b025-5669ffaff916"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:26.232522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.232455    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p" pod "a1f76beb-38b7-4318-b025-5669ffaff916" (UID: "a1f76beb-38b7-4318-b025-5669ffaff916"). InnerVolumeSpecName "kube-api-access-l9d9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:26.232675 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.232647    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs" pod "a1f76beb-38b7-4318-b025-5669ffaff916" (UID: "a1f76beb-38b7-4318-b025-5669ffaff916"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:26.331934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.331902    4536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1f76beb-38b7-4318-b025-5669ffaff916-tls-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:26.331934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.331930    4536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-dshm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:26.331934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.331940    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9d9p\" (UniqueName: \"kubernetes.io/projected/a1f76beb-38b7-4318-b025-5669ffaff916-kube-api-access-l9d9p\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:26.332160 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.331949    4536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1f76beb-38b7-4318-b025-5669ffaff916-home\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:26.842176 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.842138    4536 generic.go:356] "Generic (PLEG): container finished" podID="a1f76beb-38b7-4318-b025-5669ffaff916" containerID="6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e" exitCode=0
Oct 30 22:41:26.842737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.842187    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"
Oct 30 22:41:26.842737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.842232    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" event={"ID":"a1f76beb-38b7-4318-b025-5669ffaff916","Type":"ContainerDied","Data":"6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e"}
Oct 30 22:41:26.842737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.842276    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx" event={"ID":"a1f76beb-38b7-4318-b025-5669ffaff916","Type":"ContainerDied","Data":"b2f475ff318634c4ced45754ca0b15f6ae28796fbd2fd1859d254e86a382a85d"}
Oct 30 22:41:26.842737 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.842300    4536 scope.go:122] "RemoveContainer" containerID="6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e"
Oct 30 22:41:26.851621 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.851602    4536 scope.go:122] "RemoveContainer" containerID="6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e"
Oct 30 22:41:26.851872 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:26.851854    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e\": container with ID starting with 6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e not found: ID does not exist" containerID="6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e"
Oct 30 22:41:26.851964 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:26.851881    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e"} err="failed to get container status \"6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e\": rpc error: code = NotFound desc = could not find container \"6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e\": container with ID starting with 6881faa80308ce9fbef2f7d7c5f7375e2f7b5339a2f64a3a33e4adaf40f2153e not found: ID does not exist"
Oct 30 22:41:27.013820 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:27.013780    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"]
Oct 30 22:41:27.018625 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:27.018593    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-c76d9e/distinct-2-w6-c76d9e-kserve-855f476566-7q9lx"]
Oct 30 22:41:27.163283 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:27.163209    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" path="/var/lib/kubelet/pods/a1f76beb-38b7-4318-b025-5669ffaff916/volumes"
Oct 30 22:41:37.223047 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.223013    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"]
Oct 30 22:41:37.223563 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.223200    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" podUID="c08065b3-fe54-489e-aef6-07855e1e382f" containerName="maas-api" containerID="cri-o://84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c" gracePeriod=30
Oct 30 22:41:37.486985 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.486779    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:41:37.640736 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.640702    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm\" (UniqueName: \"kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm\") pod \"c08065b3-fe54-489e-aef6-07855e1e382f\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") "
Oct 30 22:41:37.640928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.640753    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\") pod \"c08065b3-fe54-489e-aef6-07855e1e382f\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") "
Oct 30 22:41:37.640928 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.640851    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs\") pod \"c08065b3-fe54-489e-aef6-07855e1e382f\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") "
Oct 30 22:41:37.641055 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.640958    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca\") pod \"c08065b3-fe54-489e-aef6-07855e1e382f\" (UID: \"c08065b3-fe54-489e-aef6-07855e1e382f\") "
Oct 30 22:41:37.641315 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.641285    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca" pod "c08065b3-fe54-489e-aef6-07855e1e382f" (UID: "c08065b3-fe54-489e-aef6-07855e1e382f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:37.642991 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.642961    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls" pod "c08065b3-fe54-489e-aef6-07855e1e382f" (UID: "c08065b3-fe54-489e-aef6-07855e1e382f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:37.643102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.643000    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs" pod "c08065b3-fe54-489e-aef6-07855e1e382f" (UID: "c08065b3-fe54-489e-aef6-07855e1e382f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:37.643102 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.643080    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm" pod "c08065b3-fe54-489e-aef6-07855e1e382f" (UID: "c08065b3-fe54-489e-aef6-07855e1e382f"). InnerVolumeSpecName "kube-api-access-gdzrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:37.743091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.742930    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c08065b3-fe54-489e-aef6-07855e1e382f-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:37.743091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.742975    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdzrm\" (UniqueName: \"kubernetes.io/projected/c08065b3-fe54-489e-aef6-07855e1e382f-kube-api-access-gdzrm\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:37.743091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.742991    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:37.743091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.743004    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c08065b3-fe54-489e-aef6-07855e1e382f-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:37.890907 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.890873    4536 generic.go:356] "Generic (PLEG): container finished" podID="c08065b3-fe54-489e-aef6-07855e1e382f" containerID="84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c" exitCode=0
Oct 30 22:41:37.891076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.890929    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"
Oct 30 22:41:37.891076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.890942    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" event={"ID":"c08065b3-fe54-489e-aef6-07855e1e382f","Type":"ContainerDied","Data":"84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c"}
Oct 30 22:41:37.891076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.890995    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw" event={"ID":"c08065b3-fe54-489e-aef6-07855e1e382f","Type":"ContainerDied","Data":"0b8e00a09b9068b6c9c0a02ae50095c5bd093a82ed3c5c73c7eb1716981d16d3"}
Oct 30 22:41:37.891076 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.891024    4536 scope.go:122] "RemoveContainer" containerID="84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c"
Oct 30 22:41:37.900147 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.900128    4536 scope.go:122] "RemoveContainer" containerID="84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c"
Oct 30 22:41:37.900462 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:37.900438    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c\": container with ID starting with 84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c not found: ID does not exist" containerID="84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c"
Oct 30 22:41:37.900508 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:37.900477    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c"} err="failed to get container status \"84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c\": rpc error: code = NotFound desc = could not find container \"84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c\": container with ID starting with 84c4b74016f383f4673b2b76d253b2c0f69df467fbdcb93a283b1f4f3b55f34c not found: ID does not exist"
Oct 30 22:41:38.059242 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:38.059162    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"]
Oct 30 22:41:38.064279 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:38.064255    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c76d9e-5567c9df7c-cpbfw"]
Oct 30 22:41:39.165174 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:39.165144    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c08065b3-fe54-489e-aef6-07855e1e382f" path="/var/lib/kubelet/pods/c08065b3-fe54-489e-aef6-07855e1e382f/volumes"
Oct 30 22:41:47.594253 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:47.594171    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"]
Oct 30 22:41:47.594685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:47.594419    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy" containerID="cri-o://bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145" gracePeriod=30
Oct 30 22:41:47.643545 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:47.643512    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"]
Oct 30 22:41:47.643778 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:47.643741    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy" containerID="cri-o://5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5" gracePeriod=30
Oct 30 22:41:50.805246 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:50.805209    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:41:52.887270 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.887243    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:41:52.917627 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.917602    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:41:52.960515 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.960483    4536 generic.go:356] "Generic (PLEG): container finished" podID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerID="bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145" exitCode=0
Oct 30 22:41:52.960680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.960534    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"
Oct 30 22:41:52.960680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.960586    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" event={"ID":"df0b04b6-d98a-4838-8641-2ed9778d0865","Type":"ContainerDied","Data":"bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145"}
Oct 30 22:41:52.960680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.960645    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd" event={"ID":"df0b04b6-d98a-4838-8641-2ed9778d0865","Type":"ContainerDied","Data":"0101318c9092ba766fd4a200fef6f328c69323d3d436cf5ce5a995936766edf2"}
Oct 30 22:41:52.960680 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.960677    4536 scope.go:122] "RemoveContainer" containerID="bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145"
Oct 30 22:41:52.961962 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.961943    4536 generic.go:356] "Generic (PLEG): container finished" podID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerID="5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5" exitCode=0
Oct 30 22:41:52.962060 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.961990    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"
Oct 30 22:41:52.962060 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.961992    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" event={"ID":"35b4f057-54f5-4aca-a01d-d07c1dcc7500","Type":"ContainerDied","Data":"5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5"}
Oct 30 22:41:52.962060 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.962022    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf" event={"ID":"35b4f057-54f5-4aca-a01d-d07c1dcc7500","Type":"ContainerDied","Data":"ba12b3e3a65fab0473608ce647813da84d57061eb81b493c350c3174e742c2be"}
Oct 30 22:41:52.969464 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.969446    4536 scope.go:122] "RemoveContainer" containerID="bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145"
Oct 30 22:41:52.972673 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:52.972493    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145\": container with ID starting with bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145 not found: ID does not exist" containerID="bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145"
Oct 30 22:41:52.972673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.972527    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145"} err="failed to get container status \"bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145\": rpc error: code = NotFound desc = could not find container \"bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145\": container with ID starting with bfd24e3fb9eabf97e73c51ce2ec2b97fd7c210c597cdd36616854c63cac0b145 not found: ID does not exist"
Oct 30 22:41:52.972673 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.972579    4536 scope.go:122] "RemoveContainer" containerID="5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5"
Oct 30 22:41:52.981132 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.981115    4536 scope.go:122] "RemoveContainer" containerID="5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5"
Oct 30 22:41:52.981436 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:52.981418    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5\": container with ID starting with 5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5 not found: ID does not exist" containerID="5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5"
Oct 30 22:41:52.981482 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.981445    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5"} err="failed to get container status \"5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5\": rpc error: code = NotFound desc = could not find container \"5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5\": container with ID starting with 5e6c8708a1894c80f1b7be513be01d05fd0cf712d752a70b1e7ecefa5996c9b5 not found: ID does not exist"
Oct 30 22:41:52.986699 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986676    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.986797 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986716    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.986797 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986757    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.986887 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986810    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.986887 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986849    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.986887 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986878    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986909    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.987001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986940    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987001 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.986968    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.987112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987000    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.987112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987042    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.987112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987057    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987071    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:52.987112 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987093    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987131    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.987322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987170    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987231    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy\") pod \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\" (UID: \"35b4f057-54f5-4aca-a01d-d07c1dcc7500\") "
Oct 30 22:41:52.987322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987276    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987307    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987468    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket\") pod \"df0b04b6-d98a-4838-8641-2ed9778d0865\" (UID: \"df0b04b6-d98a-4838-8641-2ed9778d0865\") "
Oct 30 22:41:52.987601 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987567    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.987787 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987765    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.988017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.987995    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:52.988118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988021    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:52.988118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988037    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:52.988118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988054    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:52.988118 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988075    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.988657 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988277    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.988657 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988432    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.988785 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988712    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.988841 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988811    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:52.988982 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.988949    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.990780 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.990029    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:41:52.990780 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.990663    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "kube-api-access-4vwjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:52.990780 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.990725    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.990780 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.990763    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:52.991092 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.991075    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:52.991251 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.991233    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo" pod "df0b04b6-d98a-4838-8641-2ed9778d0865" (UID: "df0b04b6-d98a-4838-8641-2ed9778d0865"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:41:52.991513 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.991496    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:52.991747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:52.991715    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl" pod "35b4f057-54f5-4aca-a01d-d07c1dcc7500" (UID: "35b4f057-54f5-4aca-a01d-d07c1dcc7500"). InnerVolumeSpecName "kube-api-access-c24wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:53.089522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089492    4536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089519    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089522 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089528    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089539    4536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df0b04b6-d98a-4838-8641-2ed9778d0865-istiod-ca-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089554    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089567    4536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-credential-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089581    4536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-workload-socket\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089594    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c24wl\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-kube-api-access-c24wl\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089608    4536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-data\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089622    4536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-envoy\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089636    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089649    4536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b4f057-54f5-4aca-a01d-d07c1dcc7500-istio-token\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089662    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vwjq\" (UniqueName: \"kubernetes.io/projected/df0b04b6-d98a-4838-8641-2ed9778d0865-kube-api-access-4vwjq\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.089833 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.089675    4536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df0b04b6-d98a-4838-8641-2ed9778d0865-istio-podinfo\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:53.391364 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.391327    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"]
Oct 30 22:41:53.392779 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.392755    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4gjwd"]
Oct 30 22:41:53.504293 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.504260    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"]
Oct 30 22:41:53.511100 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:53.511065    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-4fbhf"]
Oct 30 22:41:55.163523 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.163490    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" path="/var/lib/kubelet/pods/35b4f057-54f5-4aca-a01d-d07c1dcc7500/volumes"
Oct 30 22:41:55.164113 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.163902    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" path="/var/lib/kubelet/pods/df0b04b6-d98a-4838-8641-2ed9778d0865/volumes"
Oct 30 22:41:55.286213 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.286183    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-687b4b7474-jbk42"]
Oct 30 22:41:55.286434 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.286409    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-687b4b7474-jbk42" podUID="517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" containerName="manager" containerID="cri-o://e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e" gracePeriod=30
Oct 30 22:41:55.533136 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.533113    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:41:55.611287 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.611254    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8\" (UniqueName: \"kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8\") pod \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") "
Oct 30 22:41:55.611483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.611330    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert\") pod \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") "
Oct 30 22:41:55.611483 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.611441    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs\") pod \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\" (UID: \"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97\") "
Oct 30 22:41:55.613552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.613522    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert" pod "517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" (UID: "517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:55.613552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.613534    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8" pod "517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" (UID: "517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97"). InnerVolumeSpecName "kube-api-access-5bxf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:55.613693 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.613560    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs" pod "517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" (UID: "517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:41:55.713139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.713065    4536 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-webhook-cert\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:55.713139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.713089    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:55.713139 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.713100    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bxf8\" (UniqueName: \"kubernetes.io/projected/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97-kube-api-access-5bxf8\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:55.977211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.977125    4536 generic.go:356] "Generic (PLEG): container finished" podID="517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" containerID="e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e" exitCode=0
Oct 30 22:41:55.977211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.977171    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687b4b7474-jbk42" event={"ID":"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97","Type":"ContainerDied","Data":"e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e"}
Oct 30 22:41:55.977211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.977209    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687b4b7474-jbk42" event={"ID":"517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97","Type":"ContainerDied","Data":"2a6c04c3ac97f8da2e6cbeea24af8dd768613dfdd62304ff6836452db4dcf9d8"}
Oct 30 22:41:55.977542 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.977228    4536 scope.go:122] "RemoveContainer" containerID="e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e"
Oct 30 22:41:55.977542 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.977175    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687b4b7474-jbk42"
Oct 30 22:41:55.986810 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.986782    4536 scope.go:122] "RemoveContainer" containerID="e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e"
Oct 30 22:41:55.987124 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:41:55.987104    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e\": container with ID starting with e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e not found: ID does not exist" containerID="e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e"
Oct 30 22:41:55.987198 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:55.987132    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e"} err="failed to get container status \"e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e\": rpc error: code = NotFound desc = could not find container \"e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e\": container with ID starting with e97e0a85094ee419dd5bc52d9b89c1eff884394ca00619d1e202112b2ecc033e not found: ID does not exist"
Oct 30 22:41:56.144031 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:56.143999    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-687b4b7474-jbk42"]
Oct 30 22:41:56.193705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:56.193673    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-687b4b7474-jbk42"]
Oct 30 22:41:57.164810 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:41:57.164772    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" path="/var/lib/kubelet/pods/517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97/volumes"
Oct 30 22:42:17.905509 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905476    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"]
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905823    4536 cpu_manager.go:414] "containerMap: removing container" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905834    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905851    4536 cpu_manager.go:414] "containerMap: removing container" podUID="517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" containerName="manager"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905856    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" containerName="manager"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905862    4536 cpu_manager.go:414] "containerMap: removing container" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905867    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905877    4536 cpu_manager.go:414] "containerMap: removing container" podUID="87f414d1-0414-4945-8552-b2ac20bf4649" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905882    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87f414d1-0414-4945-8552-b2ac20bf4649" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905888    4536 cpu_manager.go:414] "containerMap: removing container" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905893    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905904    4536 cpu_manager.go:414] "containerMap: removing container" podUID="657220e1-3d93-4875-8a4c-b727c3e9751e" containerName="payload-pre-processing"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905909    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="657220e1-3d93-4875-8a4c-b727c3e9751e" containerName="payload-pre-processing"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905917    4536 cpu_manager.go:414] "containerMap: removing container" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905921    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905928    4536 cpu_manager.go:414] "containerMap: removing container" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905933    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905941    4536 cpu_manager.go:414] "containerMap: removing container" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905945    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905955    4536 cpu_manager.go:414] "containerMap: removing container" podUID="de107361-6bf7-462d-abd8-3ef49de8e097" containerName="payload-processing"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905960    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de107361-6bf7-462d-abd8-3ef49de8e097" containerName="payload-processing"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905966    4536 cpu_manager.go:414] "containerMap: removing container" podUID="e3ded310-d258-4da0-81e5-dc7924ca1886" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905970    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3ded310-d258-4da0-81e5-dc7924ca1886" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905977    4536 cpu_manager.go:414] "containerMap: removing container" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905981    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905986    4536 cpu_manager.go:414] "containerMap: removing container" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905990    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.905997    4536 cpu_manager.go:414] "containerMap: removing container" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906001    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906011    4536 cpu_manager.go:414] "containerMap: removing container" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906015    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906022    4536 cpu_manager.go:414] "containerMap: removing container" podUID="a8737966-b199-4970-baa9-922ee6f0b015" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906026    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8737966-b199-4970-baa9-922ee6f0b015" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906031    4536 cpu_manager.go:414] "containerMap: removing container" podUID="1b3d1f77-b03e-4553-9331-75bffb1d391a" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906035    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b3d1f77-b03e-4553-9331-75bffb1d391a" containerName="maas-api"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906040    4536 cpu_manager.go:414] "containerMap: removing container" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy"
Oct 30 22:42:17.906027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906044    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906051    4536 cpu_manager.go:414] "containerMap: removing container" podUID="c08065b3-fe54-489e-aef6-07855e1e382f" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906055    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c08065b3-fe54-489e-aef6-07855e1e382f" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906062    4536 cpu_manager.go:414] "containerMap: removing container" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906066    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906112    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1f76beb-38b7-4318-b025-5669ffaff916" containerName="main"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906119    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="5799f173-0882-4895-8049-087dfac11054" containerName="main"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906124    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="de107361-6bf7-462d-abd8-3ef49de8e097" containerName="payload-processing"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906132    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3ded310-d258-4da0-81e5-dc7924ca1886" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906137    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b5bbc31-dd7e-4d5b-8b5b-22c20ac95ffb" containerName="main"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906142    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="35b4f057-54f5-4aca-a01d-d07c1dcc7500" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906147    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="744e95b0-6cc9-4160-898e-a7aab48c54d3" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906153    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906158    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8737966-b199-4970-baa9-922ee6f0b015" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906164    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b3d1f77-b03e-4553-9331-75bffb1d391a" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906169    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="740b2afb-173f-4c0e-a22e-892cb69da226" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906176    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="c08065b3-fe54-489e-aef6-07855e1e382f" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906180    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c9a5a29-7dcf-45c7-a03b-8ae4fc906005" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906185    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906191    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="517d5cf5-0ea2-4bb2-8aa7-2e56123b8b97" containerName="manager"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906196    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="87f414d1-0414-4945-8552-b2ac20bf4649" containerName="maas-api"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906200    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="df0b04b6-d98a-4838-8641-2ed9778d0865" containerName="istio-proxy"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906205    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="657220e1-3d93-4875-8a4c-b727c3e9751e" containerName="payload-pre-processing"
Oct 30 22:42:17.907882 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.906210    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a8b6174-67fc-4776-bc8f-4a72565638ff" containerName="istio-proxy"
Oct 30 22:42:17.914127 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.913747    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.924934 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.924908    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925040 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.924955    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925090 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925061    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925144 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925117    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925195    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925219    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925246 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925240    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925258    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:17.925418 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:17.925354    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5czw\" (UniqueName: \"kubernetes.io/projected/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-kube-api-access-p5czw\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026738 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026701    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026762    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026813    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026834    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026863    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026886    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.026924 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026910    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5czw\" (UniqueName: \"kubernetes.io/projected/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-kube-api-access-p5czw\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026941    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.026982    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027307 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.027262    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027493 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.027318    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027493 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.027467    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027639 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.027613    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.027748 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.027727    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.029740 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.029714    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.029987 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.029969    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.033223 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.033202    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"]
Oct 30 22:42:18.089584 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.089554    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.089747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.089669    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5czw\" (UniqueName: \"kubernetes.io/projected/27a95f9d-0b70-4a5d-93a6-c8290b1bab48-kube-api-access-p5czw\") pod \"maas-default-gateway-openshift-default-76597d6855-mqlsc\" (UID: \"27a95f9d-0b70-4a5d-93a6-c8290b1bab48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.227210 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.227184    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:18.661340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.661311    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"]
Oct 30 22:42:18.663474 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:42:18.663446    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a95f9d_0b70_4a5d_93a6_c8290b1bab48.slice/crio-6b8e88eef85b04e84d8132190d8d6dee8b010baee570d7515ff9d637b74fa5d2 WatchSource:0}: Error finding container 6b8e88eef85b04e84d8132190d8d6dee8b010baee570d7515ff9d637b74fa5d2: Status 404 returned error can't find the container with id 6b8e88eef85b04e84d8132190d8d6dee8b010baee570d7515ff9d637b74fa5d2
Oct 30 22:42:18.665663 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.665624    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:42:18.665747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.665694    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:42:18.665747 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:18.665724    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:42:19.083685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:19.083642    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" event={"ID":"27a95f9d-0b70-4a5d-93a6-c8290b1bab48","Type":"ContainerStarted","Data":"91a2fea99548d7d2e0107a6ceb09e3f23c34775aaba25f3ae453f4c24ccb51d9"}
Oct 30 22:42:19.084193 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:19.083721    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" event={"ID":"27a95f9d-0b70-4a5d-93a6-c8290b1bab48","Type":"ContainerStarted","Data":"6b8e88eef85b04e84d8132190d8d6dee8b010baee570d7515ff9d637b74fa5d2"}
Oct 30 22:42:19.227787 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:19.227751    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:19.229269 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:19.229244    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:19.229426 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:19.229290    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:19.270235 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:19.270179    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podStartSLOduration=2.270161453 podStartE2EDuration="2.270161453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:19.267995118 +0000 UTC m=+2870.255803381" watchObservedRunningTime="2026-10-30 22:42:19.270161453 +0000 UTC m=+2870.257969719"
Oct 30 22:42:20.227811 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:20.227774    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:20.228221 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:20.227831    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:20.821582 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:42:20.821546    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:42:21.227978 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:21.227938    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:21.228489 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:21.227998    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:22.227753 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:22.227720    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:22.227939 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:22.227772    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:23.227791 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:23.227760    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:23.228189 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:23.227815    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:24.228573 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:24.228529    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:24.228983 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:24.228613    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:25.228322 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:25.228283    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 22:42:25.228531 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:25.228347    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 22:42:27.228797 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:27.228756    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:42:27.229274 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:27.228815    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:42:28.269538 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:28.269499    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:42:28.270004 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:28.269551    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:42:28.798178 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:28.798145    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:28.798445 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:28.798425    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:28.799078 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:28.799061    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc"
Oct 30 22:42:33.026149 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.026115    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"]
Oct 30 22:42:33.026600 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.026585    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="faef18ca-2ab9-4f4d-9611-7e7b49314f18" containerName="istio-proxy"
Oct 30 22:42:33.030849 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.030833    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187410 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187362    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187432    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187522    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187586 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187555    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldgnd\" (UniqueName: \"kubernetes.io/projected/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-kube-api-access-ldgnd\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187707 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187589    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187707 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187678    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187707    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187794 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187724    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.187888 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.187805    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.241870 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.241835    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"]
Oct 30 22:42:33.289045 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.288971    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289045 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289011    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289045 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289042    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289071    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289125    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289357 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289265    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289368    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289445    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-data\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289455    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289495    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-workload-certs\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289552 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289503    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldgnd\" (UniqueName: \"kubernetes.io/projected/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-kube-api-access-ldgnd\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289910 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289889    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.289973 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.289921    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-workload-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.290260 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.290237    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-credential-socket\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.291893 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.291873    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-envoy\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.291944 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.291887    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.353682 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.353656    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-istio-token\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.353958 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.353936    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldgnd\" (UniqueName: \"kubernetes.io/projected/6836e64a-4b13-48eb-9cc8-0fb328a66dbb-kube-api-access-ldgnd\") pod \"maas-default-gateway-openshift-default-76597d6855-47kdc\" (UID: \"6836e64a-4b13-48eb-9cc8-0fb328a66dbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:33.643595 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:33.643516    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:34.073540 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.073511    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"]
Oct 30 22:42:34.075348 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:42:34.075321    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6836e64a_4b13_48eb_9cc8_0fb328a66dbb.slice/crio-d7b2eadaa72c21ddac9df0751fc49885e03e9a4eea860157d54d33b01cea8f71 WatchSource:0}: Error finding container d7b2eadaa72c21ddac9df0751fc49885e03e9a4eea860157d54d33b01cea8f71: Status 404 returned error can't find the container with id d7b2eadaa72c21ddac9df0751fc49885e03e9a4eea860157d54d33b01cea8f71
Oct 30 22:42:34.077661 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.077631    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:42:34.077720 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.077696    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:42:34.077765 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.077724    4536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892000Ki","pods":"250"}
Oct 30 22:42:34.166777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.166742    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" event={"ID":"6836e64a-4b13-48eb-9cc8-0fb328a66dbb","Type":"ContainerStarted","Data":"234cd347fa10cb4904d25df99f322efbb7f360179482427bd831d59e110ebb00"}
Oct 30 22:42:34.166952 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.166797    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" event={"ID":"6836e64a-4b13-48eb-9cc8-0fb328a66dbb","Type":"ContainerStarted","Data":"d7b2eadaa72c21ddac9df0751fc49885e03e9a4eea860157d54d33b01cea8f71"}
Oct 30 22:42:34.290211 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.290147    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podStartSLOduration=2.29013227 podStartE2EDuration="2.29013227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:34.289192429 +0000 UTC m=+2885.277000693" watchObservedRunningTime="2026-10-30 22:42:34.29013227 +0000 UTC m=+2885.277940533"
Oct 30 22:42:34.644018 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:34.643974    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:35.646143 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:35.646111    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:42:35.646610 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:35.646166    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:42:36.644106 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:36.644070    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:42:36.644292 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:36.644126    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:42:37.644253 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:37.644212    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:42:37.644728 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:37.644267    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:42:38.644705 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:38.644665    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:42:38.645177 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:38.644728    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:42:39.643994 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:39.643959    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:42:39.644178 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:39.644010    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:42:40.644144 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:40.644109    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:42:40.644577 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:40.644164    4536 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:42:42.287064 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:42.287025    4536 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:42.287614 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:42.287373    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:42.288340 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:42:42.288321    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc"
Oct 30 22:42:50.804003 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:42:50.803964    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:43:20.860558 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:43:20.860464    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:43:50.805440 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:43:50.805381    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:44:20.804263 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:44:20.804215    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:44:50.804233 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:44:50.804144    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:45:20.805681 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:45:20.805642    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:45:50.804160 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:45:50.804120    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:46:20.805338 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:46:20.805248    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:46:28.345906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:46:28.345500    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:46:28.345906 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:46:28.345563    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:46:50.808224 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:46:50.807125    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:46:54.707914 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:46:54.707502    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-tqtgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:46:54.707914 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:46:54.707585    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-tqtgt" podUID="cf41528b-55be-45e7-9624-14885716b61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:47:20.805633 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:47:20.805587    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:47:29.844286 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:47:29.844227    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:47:29.844992 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:47:29.844310    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:47:50.804927 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:47:50.804844    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:48:20.805162 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:48:20.805126    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:48:29.846604 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:48:29.846563    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-mqlsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:48:29.847252 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:48:29.847203    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-mqlsc" podUID="27a95f9d-0b70-4a5d-93a6-c8290b1bab48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:48:50.804901 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:48:50.804863    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:49:20.803879 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:49:20.803802    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:49:50.805824 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:49:50.805790    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:50:20.805641 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:50:20.805604    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:50:50.804057 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:50:50.803965    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:51:20.804762 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:51:20.804727    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:51:50.803949 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:51:50.803911    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:52:20.804002 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:52:20.803924    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:52:50.804430 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:52:50.804375    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:53:20.804841 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:53:20.804800    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:53:50.804840 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:53:50.804799    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:54:20.806238 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:54:20.806202    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:54:50.803892 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:54:50.803856    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:55:20.805662 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:55:20.805580    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:55:50.803997 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:55:50.803961    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:56:20.803971 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:56:20.803934    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:56:28.331017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:56:28.330516    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:56:28.331017 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:56:28.330585    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:56:50.804655 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:56:50.804551    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:57:20.805648 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:57:20.805612    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:57:50.807040 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:57:50.807004    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:58:20.807629 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:58:20.807545    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:58:50.807072 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:58:50.807033    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:59:06.196452 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.196415    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"]
Oct 30 22:59:06.199922 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.199902    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.303970 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.303941    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"]
Oct 30 22:59:06.385467 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.385434    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7914bfaf-f819-45a8-a512-c540c6997bf2-openshift-service-ca\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.385643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.385476    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7914bfaf-f819-45a8-a512-c540c6997bf2-metrics-certs\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.385643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.385498    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkhwh\" (UniqueName: \"kubernetes.io/projected/7914bfaf-f819-45a8-a512-c540c6997bf2-kube-api-access-xkhwh\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.385643 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.385597    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7914bfaf-f819-45a8-a512-c540c6997bf2-maas-api-tls\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.486453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.486350    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7914bfaf-f819-45a8-a512-c540c6997bf2-openshift-service-ca\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.486453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.486390    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7914bfaf-f819-45a8-a512-c540c6997bf2-metrics-certs\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.486453 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.486438    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkhwh\" (UniqueName: \"kubernetes.io/projected/7914bfaf-f819-45a8-a512-c540c6997bf2-kube-api-access-xkhwh\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.486686 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.486464    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7914bfaf-f819-45a8-a512-c540c6997bf2-maas-api-tls\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.486979 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.486947    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7914bfaf-f819-45a8-a512-c540c6997bf2-openshift-service-ca\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.489244 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.489223    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7914bfaf-f819-45a8-a512-c540c6997bf2-maas-api-tls\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.489316 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.489223    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7914bfaf-f819-45a8-a512-c540c6997bf2-metrics-certs\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.549181 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.549152    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkhwh\" (UniqueName: \"kubernetes.io/projected/7914bfaf-f819-45a8-a512-c540c6997bf2-kube-api-access-xkhwh\") pod \"maas-api-768d68dfcd-5d2c9\" (UID: \"7914bfaf-f819-45a8-a512-c540c6997bf2\") " pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:06.813301 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:06.813217    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:07.191625 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:07.191600    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"]
Oct 30 22:59:07.193468 ip-10-0-138-166 kubenswrapper[4536]: W1030 22:59:07.193434    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7914bfaf_f819_45a8_a512_c540c6997bf2.slice/crio-818a6cbebb2852a88fa7bcbba4be05476ff01195d55e337ecb374705829fde3b WatchSource:0}: Error finding container 818a6cbebb2852a88fa7bcbba4be05476ff01195d55e337ecb374705829fde3b: Status 404 returned error can't find the container with id 818a6cbebb2852a88fa7bcbba4be05476ff01195d55e337ecb374705829fde3b
Oct 30 22:59:07.194777 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:07.194760    4536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:59:07.759215 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:07.759178    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9" event={"ID":"7914bfaf-f819-45a8-a512-c540c6997bf2","Type":"ContainerStarted","Data":"818a6cbebb2852a88fa7bcbba4be05476ff01195d55e337ecb374705829fde3b"}
Oct 30 22:59:08.765532 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:08.765499    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9" event={"ID":"7914bfaf-f819-45a8-a512-c540c6997bf2","Type":"ContainerStarted","Data":"77fd84a18bf5b20b97505f189499a1dd2694256f071aad654632f994a48379c2"}
Oct 30 22:59:08.766063 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:08.765628    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:08.886897 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:08.886844    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9" podStartSLOduration=2.094189114 podStartE2EDuration="2.886830158s" totalImagesPullingTime="792.641044ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:08.885524078 +0000 UTC m=+3879.873332354" watchObservedRunningTime="2026-10-30 22:59:08.886830158 +0000 UTC m=+3879.874638420"
Oct 30 22:59:14.775333 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:14.775302    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-768d68dfcd-5d2c9"
Oct 30 22:59:14.954851 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:14.954818    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"]
Oct 30 22:59:14.955068 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:14.955043    4536 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl" podUID="0a0d67b8-579b-446b-b8cd-803547e31e55" containerName="maas-api" containerID="cri-o://91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed" gracePeriod=30
Oct 30 22:59:16.189991 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.189968    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:59:16.290492 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.286101    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t\" (UniqueName: \"kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t\") pod \"0a0d67b8-579b-446b-b8cd-803547e31e55\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") "
Oct 30 22:59:16.290492 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.286176    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca\") pod \"0a0d67b8-579b-446b-b8cd-803547e31e55\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") "
Oct 30 22:59:16.290492 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.286240    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs\") pod \"0a0d67b8-579b-446b-b8cd-803547e31e55\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") "
Oct 30 22:59:16.290492 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.286326    4536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls\") pod \"0a0d67b8-579b-446b-b8cd-803547e31e55\" (UID: \"0a0d67b8-579b-446b-b8cd-803547e31e55\") "
Oct 30 22:59:16.291875 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.291516    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca" pod "0a0d67b8-579b-446b-b8cd-803547e31e55" (UID: "0a0d67b8-579b-446b-b8cd-803547e31e55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:59:16.295693 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.295653    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls" pod "0a0d67b8-579b-446b-b8cd-803547e31e55" (UID: "0a0d67b8-579b-446b-b8cd-803547e31e55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:59:16.297593 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.297556    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs" pod "0a0d67b8-579b-446b-b8cd-803547e31e55" (UID: "0a0d67b8-579b-446b-b8cd-803547e31e55"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:59:16.297706 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.297586    4536 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t" pod "0a0d67b8-579b-446b-b8cd-803547e31e55" (UID: "0a0d67b8-579b-446b-b8cd-803547e31e55"). InnerVolumeSpecName "kube-api-access-d8j7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:59:16.387359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.387322    4536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a0d67b8-579b-446b-b8cd-803547e31e55-openshift-service-ca\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:16.387359 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.387357    4536 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-metrics-certs\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:16.387617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.387373    4536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a0d67b8-579b-446b-b8cd-803547e31e55-maas-api-tls\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:16.387617 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.387386    4536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8j7t\" (UniqueName: \"kubernetes.io/projected/0a0d67b8-579b-446b-b8cd-803547e31e55-kube-api-access-d8j7t\") on node \"ip-10-0-138-166.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:59:16.802358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.802320    4536 generic.go:356] "Generic (PLEG): container finished" podID="0a0d67b8-579b-446b-b8cd-803547e31e55" containerID="91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed" exitCode=0
Oct 30 22:59:16.802358 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.802360    4536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"
Oct 30 22:59:16.802580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.802420    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl" event={"ID":"0a0d67b8-579b-446b-b8cd-803547e31e55","Type":"ContainerDied","Data":"91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed"}
Oct 30 22:59:16.802580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.802473    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl" event={"ID":"0a0d67b8-579b-446b-b8cd-803547e31e55","Type":"ContainerDied","Data":"cfece5cc18263b01d34ba082085fd08cd493273500087463281aaf0cf567af91"}
Oct 30 22:59:16.802580 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.802497    4536 scope.go:122] "RemoveContainer" containerID="91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed"
Oct 30 22:59:16.811685 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.811658    4536 scope.go:122] "RemoveContainer" containerID="91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed"
Oct 30 22:59:16.811950 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:59:16.811931    4536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed\": container with ID starting with 91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed not found: ID does not exist" containerID="91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed"
Oct 30 22:59:16.812027 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.811962    4536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed"} err="failed to get container status \"91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed\": rpc error: code = NotFound desc = could not find container \"91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed\": container with ID starting with 91e72812817695b029f5d9cbb27406e2757f11a5827d5c505fdf908ff034fbed not found: ID does not exist"
Oct 30 22:59:16.968477 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:16.968447    4536 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"]
Oct 30 22:59:17.018091 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:17.018055    4536 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d58dc998-8k2cl"]
Oct 30 22:59:17.163300 ip-10-0-138-166 kubenswrapper[4536]: I1030 22:59:17.163227    4536 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a0d67b8-579b-446b-b8cd-803547e31e55" path="/var/lib/kubelet/pods/0a0d67b8-579b-446b-b8cd-803547e31e55/volumes"
Oct 30 22:59:20.806599 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:59:20.806569    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 22:59:50.804040 ip-10-0-138-166 kubenswrapper[4536]: E1030 22:59:50.803956    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:00:20.804728 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:00:20.804690    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:00:50.805114 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:00:50.805077    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:01:20.805285 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:01:20.805196    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:01:50.803826 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:01:50.803789    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:02:20.804778 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:02:20.804740    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:02:50.804677 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:02:50.804587    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:03:20.805207 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:03:20.805166    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:03:43.319558 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:43.319514    4536 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76597d6855-47kdc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 23:03:43.320079 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:43.319596    4536 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76597d6855-47kdc" podUID="6836e64a-4b13-48eb-9cc8-0fb328a66dbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 23:03:50.804649 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:03:50.804612    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:03:52.380697 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:52.380669    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-cfgmp_28aac9af-8737-449d-abd5-2de2057682c1/manager/0.log"
Oct 30 23:03:52.664340 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:52.664271    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jhpf7_3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25/manager/0.log"
Oct 30 23:03:52.989720 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:52.989696    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ntktz_acd884c1-1fa7-4900-8bbc-acead715fb0e/manager/0.log"
Oct 30 23:03:54.385453 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:54.385422    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-h7wbr_cf440c32-3c0d-4220-a26e-f66f3aed1d9c/proxy/0.log"
Oct 30 23:03:55.270571 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:55.270526    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-768d68dfcd-5d2c9_7914bfaf-f819-45a8-a512-c540c6997bf2/maas-api/0.log"
Oct 30 23:03:56.630995 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:03:56.630965    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-zbxqb_939c3d2e-da04-4bc6-a11a-bc3e008d14ce/postgres/0.log"
Oct 30 23:04:01.019816 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:01.019782    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-rzrlc_8f7c83fe-c33a-4512-9fb8-859c4eab9066/discovery/0.log"
Oct 30 23:04:01.622856 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:01.622825    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-679b756675-lwmdb_7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4/kube-auth-proxy/0.log"
Oct 30 23:04:01.855974 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:01.855947    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-47kdc_6836e64a-4b13-48eb-9cc8-0fb328a66dbb/istio-proxy/0.log"
Oct 30 23:04:02.613985 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:02.613953    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-mqlsc_27a95f9d-0b70-4a5d-93a6-c8290b1bab48/istio-proxy/0.log"
Oct 30 23:04:03.372190 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:03.372163    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-tqtgt_cf41528b-55be-45e7-9624-14885716b61c/istio-proxy/0.log"
Oct 30 23:04:19.066647 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:19.066568    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-cfgmp_28aac9af-8737-449d-abd5-2de2057682c1/manager/0.log"
Oct 30 23:04:19.345633 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:19.345553    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jhpf7_3ff5916a-58a7-4db2-8ff5-02ca4cdb6d25/manager/0.log"
Oct 30 23:04:19.664846 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:19.664772    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-ntktz_acd884c1-1fa7-4900-8bbc-acead715fb0e/manager/0.log"
Oct 30 23:04:20.805581 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:04:20.805540    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:04:20.875152 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:20.875114    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-h7wbr_cf440c32-3c0d-4220-a26e-f66f3aed1d9c/proxy/0.log"
Oct 30 23:04:21.475262 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:21.475238    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-768d68dfcd-5d2c9_7914bfaf-f819-45a8-a512-c540c6997bf2/maas-api/0.log"
Oct 30 23:04:22.745316 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:22.745288    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-zbxqb_939c3d2e-da04-4bc6-a11a-bc3e008d14ce/postgres/0.log"
Oct 30 23:04:26.519607 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:26.519578    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-rzrlc_8f7c83fe-c33a-4512-9fb8-859c4eab9066/discovery/0.log"
Oct 30 23:04:27.107898 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:27.107856    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-679b756675-lwmdb_7591f97c-1ad1-4a4f-abfd-1cd5bcc121a4/kube-auth-proxy/0.log"
Oct 30 23:04:27.336239 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:27.336211    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-47kdc_6836e64a-4b13-48eb-9cc8-0fb328a66dbb/istio-proxy/0.log"
Oct 30 23:04:28.140529 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:28.140470    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-mqlsc_27a95f9d-0b70-4a5d-93a6-c8290b1bab48/istio-proxy/0.log"
Oct 30 23:04:28.964814 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:04:28.964778    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76597d6855-tqtgt_cf41528b-55be-45e7-9624-14885716b61c/istio-proxy/0.log"
Oct 30 23:04:50.804374 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:04:50.804337    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:05:20.804034 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:05:20.803995    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:05:50.804329 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:05:50.804250    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:05:51.873104 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:05:51.873067    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2xm8j_d64e1a7d-50f4-4f15-ab5c-71921ae2af7e/global-pull-secret-syncer/0.log"
Oct 30 23:05:55.398658 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:05:55.398621    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pjnv8_46124db8-6b86-4a74-8ef7-80a2a04a8463/konnectivity-agent/0.log"
Oct 30 23:05:56.316071 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:05:56.316040    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-166.us-west-1.compute.internal_c91ce82603c0eba2cf2b8be390abf9d9/haproxy/0.log"
Oct 30 23:06:01.383548 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.383512    4536 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"]
Oct 30 23:06:01.383954 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.383871    4536 cpu_manager.go:414] "containerMap: removing container" podUID="0a0d67b8-579b-446b-b8cd-803547e31e55" containerName="maas-api"
Oct 30 23:06:01.383954 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.383882    4536 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a0d67b8-579b-446b-b8cd-803547e31e55" containerName="maas-api"
Oct 30 23:06:01.384034 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.383972    4536 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a0d67b8-579b-446b-b8cd-803547e31e55" containerName="maas-api"
Oct 30 23:06:01.386978 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.386958    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.476082 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.476049    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"]
Oct 30 23:06:01.531070 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.531037    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qhgmw\"/\"openshift-service-ca.crt\""
Oct 30 23:06:01.552384 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.552358    4536 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qhgmw\"/\"default-dockercfg-zvzwl\""
Oct 30 23:06:01.552384 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.552358    4536 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qhgmw\"/\"kube-root-ca.crt\""
Oct 30 23:06:01.570322 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.570296    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-lib-modules\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.570466 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.570348    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7dfc\" (UniqueName: \"kubernetes.io/projected/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-kube-api-access-c7dfc\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.570560 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.570464    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-proc\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.570560 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.570494    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-podres\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.570560 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.570529    4536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-sys\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.671911 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.671806    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-sys\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.671911 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.671890    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-lib-modules\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672118 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.671941    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-sys\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672118 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.671955    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7dfc\" (UniqueName: \"kubernetes.io/projected/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-kube-api-access-c7dfc\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672118 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.672018    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-lib-modules\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672118 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.672022    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-proc\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672118 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.672058    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-proc\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672118 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.672063    4536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-podres\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.672310 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.672139    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-podres\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.734930 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.734904    4536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7dfc\" (UniqueName: \"kubernetes.io/projected/8d2a8def-c17b-45f9-b845-49bbcdcfb73e-kube-api-access-c7dfc\") pod \"perf-node-gather-daemonset-nwbx8\" (UID: \"8d2a8def-c17b-45f9-b845-49bbcdcfb73e\") " pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:01.998622 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:01.998589    4536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:02.372293 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:02.372266    4536 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"]
Oct 30 23:06:02.374693 ip-10-0-138-166 kubenswrapper[4536]: W1030 23:06:02.374666    4536 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8d2a8def_c17b_45f9_b845_49bbcdcfb73e.slice/crio-735e6fe05f591ab286e8b5ec395926de39f64119d0cac866505c350b16fed8ca WatchSource:0}: Error finding container 735e6fe05f591ab286e8b5ec395926de39f64119d0cac866505c350b16fed8ca: Status 404 returned error can't find the container with id 735e6fe05f591ab286e8b5ec395926de39f64119d0cac866505c350b16fed8ca
Oct 30 23:06:02.608578 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:02.608537    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8" event={"ID":"8d2a8def-c17b-45f9-b845-49bbcdcfb73e","Type":"ContainerStarted","Data":"449a60e49ba6710907c996c527d6316fe7cdcc727120be9e7543637f04c650be"}
Oct 30 23:06:02.608979 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:02.608588    4536 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8" event={"ID":"8d2a8def-c17b-45f9-b845-49bbcdcfb73e","Type":"ContainerStarted","Data":"735e6fe05f591ab286e8b5ec395926de39f64119d0cac866505c350b16fed8ca"}
Oct 30 23:06:02.608979 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:02.608660    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:02.728319 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:02.728259    4536 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8" podStartSLOduration=1.728243698 podStartE2EDuration="1.728243698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 23:06:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:06:02.727623138 +0000 UTC m=+4293.715431402" watchObservedRunningTime="2026-10-30 23:06:02.728243698 +0000 UTC m=+4293.716051961"
Oct 30 23:06:06.066843 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:06.066813    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/alertmanager/0.log"
Oct 30 23:06:06.483710 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:06.483684    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/config-reloader/0.log"
Oct 30 23:06:06.902186 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:06.902113    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/kube-rbac-proxy-web/0.log"
Oct 30 23:06:07.321593 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:07.321567    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/kube-rbac-proxy/0.log"
Oct 30 23:06:07.753960 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:07.753932    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/kube-rbac-proxy-metric/0.log"
Oct 30 23:06:08.172488 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:08.172383    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/prom-label-proxy/0.log"
Oct 30 23:06:08.590711 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:08.590684    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b3f2e36-f4a8-4f87-8c35-c8da2c5cd3a9/init-config-reloader/0.log"
Oct 30 23:06:08.623462 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:08.623438    4536 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qhgmw/perf-node-gather-daemonset-nwbx8"
Oct 30 23:06:10.845873 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:10.845845    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d94746fd5-q662f_c91e5088-18a7-4ae9-8274-0943c3a512be/metrics-server/0.log"
Oct 30 23:06:11.269001 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:11.268973    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-scd5x_1e5493e9-c44c-4903-bb43-007730d23b5f/monitoring-plugin/0.log"
Oct 30 23:06:11.704386 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:11.704360    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7hbfw_39b94723-b921-47ff-96f9-f2a9b948f92c/node-exporter/0.log"
Oct 30 23:06:12.126643 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:12.126574    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7hbfw_39b94723-b921-47ff-96f9-f2a9b948f92c/kube-rbac-proxy/0.log"
Oct 30 23:06:12.544607 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:12.544579    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7hbfw_39b94723-b921-47ff-96f9-f2a9b948f92c/init-textfile/0.log"
Oct 30 23:06:15.602428 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:15.602386    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-f5f7m_fcda885e-673f-4be4-bf9e-7a93d667ed5b/kube-rbac-proxy-main/0.log"
Oct 30 23:06:16.018960 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:16.018929    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-f5f7m_fcda885e-673f-4be4-bf9e-7a93d667ed5b/kube-rbac-proxy-self/0.log"
Oct 30 23:06:16.479979 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:16.479946    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-f5f7m_fcda885e-673f-4be4-bf9e-7a93d667ed5b/openshift-state-metrics/0.log"
Oct 30 23:06:20.805567 ip-10-0-138-166 kubenswrapper[4536]: E1030 23:06:20.805524    4536 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-r28cn" containerName="extractor"
Oct 30 23:06:21.121451 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:21.121360    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76bc55d459-9w2j9_04634017-52b0-44dc-9cb2-a75785a88081/telemeter-client/0.log"
Oct 30 23:06:21.540150 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:21.540118    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76bc55d459-9w2j9_04634017-52b0-44dc-9cb2-a75785a88081/reload/0.log"
Oct 30 23:06:21.958673 ip-10-0-138-166 kubenswrapper[4536]: I1030 23:06:21.958649    4536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76bc55d459-9w2j9_04634017-52b0-44dc-9cb2-a75785a88081/kube-rbac-proxy/0.log"